Forum Discussion

RouG's avatar
RouG
Occasional Contributor
14 years ago

Problem with Conversion

Hello,

I'm running TC 8.10.487.7 on Windows XPSP3.



I've a problem with the following code.



----------------------------------

flLastSold = aqConvert.VarToFloat(GetPersonnalAccountSold(strPID,strConnectionString))


resp_Status = WebServices.WsForAdminApps_Epurse.ChargeAccount(fltAmount,strCardId,intConfId,intOperationType)


fltNewSold = aqConvert.VarToFloat(GetPersonnalAccountSold(strPID,strConnectionString))


 


if ((fltNewSold - flLastSold) <> fltAmount) then


Log.Warning("Problem with Sold. New sold ("&GetPersonalAccountSold(strPID,strConnectionString)&") - Oldsold ("&flLastSold&") <> Amout ("&fltAmount&") loaded")


 


else


Log.Message("Sold OK")


End If



---------------------------------------

In the attached Case01.jpg , you can see that when fltAmount = 10.01, the If statement is evaluated to True even thoug everything is correct -> Not Correct

In the attached Case02.jpg , you can see that when fltAmount  = 11, the If statment is evaluated to False. -> Correct.



My regional settings are set to SwissFrench -> the Decimal separator is the ".".



So i cannot see where is the problem.

Any helps would be appriciate.



Best Regards.

   

  • RouG's avatar
    RouG
    Occasional Contributor
    14 years ago
    Hi.



    i've create a simple test:




    Public sub Test2


     Test2

    Dim tt,ti,tu


    tu = 30.02


    ti = 20.01


    tt = 10.01


    if (tu-ti-tt) then


    Log.Message("True")


    Log.Message("tu = "&tu&", ti = "&ti&", tu-ti-tt = "&tu-ti-tt)


    else


    Log.Message("False")


    Log.Message("tu = "&tu&", ti = "&ti&", tu-ti-tt = "&tu-ti-tt)


    end If

    End sub



    I should get

    False

    tu = 30.02, ti = 20.01, tu-ti-tt = 0



    What I get is :

    True   

    tu = 30.02, ti = 20.01, tu-ti-tt = -1.77635683940025E-15



    And after that i change



    tu = 20.02

    ti = 10.01

    tt = 10.01



    and this time i get what I'm expecting

    False  

    tu = 20.02, ti = 10.01, tu-ti-tt = 0  







    Can somebody run this simple test and see if he gets the same result as me please.





    Thanks.


  • AlexKaras's avatar
    AlexKaras
    Champion Level 3
    14 years ago
    Hi Gaetan,



    You are observing the pecularities of the floating point arithmetic that is always approximate but not exact (unless implemented using special data structures and algorithms (like mone data type in some languages)).

    I would suggest consider the use the Round() function.
  • RouG's avatar
    RouG
    Occasional Contributor
    14 years ago
    Hello Alexei.



    Thanks for the hint. I will give a try.



    But I'm a bit worried about  "rounding" and the fact that this might cause the  failure or passed a test case. I would prefer working on the "true" value.



    Regards.