Forum Discussion

msaunders's avatar
msaunders
Contributor
10 years ago
Solved

Check point failing with matching values

Hi all, I have a checkpoint that is checking that the value is properly stored in an infragistics grid. it seems to fail at certain times through the test. I am stepping the value up by .1 at a...
  • HKosova's avatar
    HKosova
    10 years ago
    Hi Mathew,



    You're right, it happens because of floating point math specifics. The proper way to compare floating-point numbers is within some very small tolerance, like this:



    // Replace aqObject.CheckProperty with this:

    var diff = Math.abs(ultraGridCLT.wValue(1, "Scale") - i);

    if (diff < 0.000001)

      Log.Checkpoint("The values match.");

    else

      Log.Error("The values are different.");

HKosova's avatar
HKosova
SmartBear Alumni (Retired)
10 years ago
Hi Mathew,



You're right, it happens because of floating point math specifics. The proper way to compare floating-point numbers is within some very small tolerance, like this:



// Replace aqObject.CheckProperty with this:

var diff = Math.abs(ultraGridCLT.wValue(1, "Scale") - i);

if (diff < 0.000001)

  Log.Checkpoint("The values match.");

else

  Log.Error("The values are different.");