Forum Discussion

msaunders's avatar
msaunders
Contributor
11 years ago

Check point failing with matching values

Hi all, I have a checkpoint that is checking that the value is properly stored in an infragistics grid. it seems to fail at certain times through the test. I am stepping the value up by .1 at a...
  • HKosova's avatar
    HKosova
    11 years ago
    Hi Mathew,



    You're right, it happens because of floating point math specifics. The proper way to compare floating-point numbers is within some very small tolerance, like this:



    // Replace aqObject.CheckProperty with this:

    var diff = Math.abs(ultraGridCLT.wValue(1, "Scale") - i);

    if (diff < 0.000001)

      Log.Checkpoint("The values match.");

    else

      Log.Error("The values are different.");

msaunders's avatar
msaunders
Contributor
11 years ago

If it helps this is the actual function i am using. 



function iterateScale(ultraGridCLT)


{


  for (i = 0.1; i <= 10; i += 0.1){


    var textCLT = ultraGridCLT.ultraGrid1_EmbeddableTextBox;


    //set the scale


    ultraGridCLT.ClickCellXY(1, "Scale", 66, 8);


    textCLT.SetText( i );


    windowPlaceHolder.Click(63, 305);


    ultraGridCLT.Click(268, 948);


    //check that the value is really right


    aqObject.CheckProperty(ultraGridCLT, "wValue(1, \"Scale\")", cmpEqual, i );


  }


}