Forum Discussion

testertesting's avatar
testertesting
Occasional Contributor
5 years ago

Any good ways for comparing two floats?

HI,

 

I am reading cells from two different excel files and comparing the values in them.

 

For one cell, the values I am comparing are 399.99999998 and 399.99999999.

 

I would like to compare both numbers, and if the difference is greater than 0.0000001, ignore the difference.

 

The problem I am having is, when I get the ABS of num1 - num2, this gives me 1.0000007933E-8.

 

Is there any good way to compare floats without getting E numbers? I am using vbscript.

 

Thanks in advance! 🙂

  • I found a solution.

     

    I rounded down both numbers to the acceptable decimal places before comparing. 🙂

  • Wamboo's avatar
    Wamboo
    Community Hero

    Hi, which programming language do you use?

     

    Ahhhhh. I didn't notice you wrote about vbscript.

     

    I'm sorry I don't know VB. Maybe someone else could help you.

    • testertesting's avatar
      testertesting
      Occasional Contributor

      I found a solution.

       

      I rounded down both numbers to the acceptable decimal places before comparing. 🙂