Forum Discussion

JohanH's avatar
JohanH
Occasional Contributor
10 years ago

Casting lager float numbers to int does not work when using python

I'm having issues casting a large float to int in python. It works fine when running python from the console but not in TestComplete. I need to do some bit operations, using a timestamp as base. ...
  • joseph_michaud's avatar
    joseph_michaud
    10 years ago

    I think you may be seeing a bug with how TestComplete is converting python variables.  If you convert any of the variables to a string and print out the string, the variable seems to have the correct value.  From your example:

     

     

    import datetime
import math

def test():
  dt = datetime.datetime(2015,10,30,12,29,5,987)
  ms = dt.timestamp() * 1000 * 1000
  Log.Message("aqConvert.FloatToStr(ms) = " + aqConvert.FloatToStr(ms))
  Log.Message("str(ms) = " + str(ms))
  
  intms = int(ms)
  Log.Message("aqConvert.IntToStr(intms) = " + aqConvert.IntToStr(intms))
  Log.Message("str(intms) = " + str(intms))

     

    produces this output

     

    aqConvert.FloatToStr(ms) = 1.44622254500099E+15
str(ms) = 1446222545000987.0
aqConvert.IntToStr(intms) = -1
str(intms) = 1446222545000987

    When debugging, the local variable also (incorrectly) shows the intms value as -1.

     

    I haven't pursued this much further, but I think your calculations will be OK and that you should use the Python string conversion to look at your values for the time being.

     

joseph_michaud's avatar
joseph_michaud
Moderator

I think you may be seeing a bug with how TestComplete is converting python variables.  If you convert any of the variables to a string and print out the string, the variable seems to have the correct value.  From your example:

 

 

import datetime
import math

def test():
  dt = datetime.datetime(2015,10,30,12,29,5,987)
  ms = dt.timestamp() * 1000 * 1000
  Log.Message("aqConvert.FloatToStr(ms) = " + aqConvert.FloatToStr(ms))
  Log.Message("str(ms) = " + str(ms))
  
  intms = int(ms)
  Log.Message("aqConvert.IntToStr(intms) = " + aqConvert.IntToStr(intms))
  Log.Message("str(intms) = " + str(intms))

 

produces this output

 

aqConvert.FloatToStr(ms) = 1.44622254500099E+15
str(ms) = 1446222545000987.0
aqConvert.IntToStr(intms) = -1
str(intms) = 1446222545000987

When debugging, the local variable also (incorrectly) shows the intms value as -1.

 

I haven't pursued this much further, but I think your calculations will be OK and that you should use the Python string conversion to look at your values for the time being.

 

JohanH's avatar
JohanH
Occasional Contributor
10 years ago

Thanks for your effort.

 

I do however believe that making calculations in the value did not work... at least not the bitwise operations but I'll give it another spin and see how it turns out...