Ask a Question

Any good ways for comparing two floats?

SOLVED
testertesting
Occasional Contributor

Any good ways for comparing two floats?

HI,

 

I am reading cells from two different excel files and comparing the values in them.

 

For one cell, the values I am comparing are 399.99999998 and 399.99999999.

 

I would like to compare both numbers, and if the difference is greater than 0.0000001, ignore the difference.

 

The problem I am having is, when I get the ABS of num1 - num2, this gives me 1.0000007933E-8.

 

Is there any good way to compare floats without getting E numbers? I am using vbscript.

 

Thanks in advance! 🙂

2 REPLIES 2
Wamboo
Community Hero

Hi, which programming language do you use?

 

Ahhhhh. I didn't notice you wrote about vbscript.

 

I'm sorry I don't know VB. Maybe someone else could help you.

testertesting
Occasional Contributor

I found a solution.

 

I rounded down both numbers to the acceptable decimal places before comparing. 🙂

cancel
Showing results for 
Search instead for 
Did you mean: