Forum Discussion

sastowe's avatar
sastowe
Super Contributor
13 years ago

two doubles not equal when they really are

I have a procedure that I am running for three different test cases.  For each test case, I pass a string of expected values. The string is split into an array. I am comparing the data in my database using an ADO recordset to the expected good values.



The first two test cases work fine. In the third test case, each time I evaluate



If rs("ResultValue").Value <> cdbl(asValues(i)) then



This is returning true when the values sure as heck look the same to me. In debugging, I expressly cast each to a double to make sure we are not seeing some data type vagary.

2 Replies

  • sastowe's avatar
    sastowe
    Super Contributor
    13 years ago
    Sorry I was not done but posted the pic.



    Looking at the debug window, they look equal to me.



    This is a simple logical operation. I must be doing something dumb. But I can't for the life of me imagine what.



    Any thoughts?
  • sastowe's avatar
    sastowe
    Super Contributor
    13 years ago
    So I did some troubleshooting. Seems the difference between the test cases that work and the ones that don't are the number of decimal places in the compared values. I mentioned this to my mega geek husband who tells me that this can be the case for values that are defined as type DOUBLE. I don't claim to understand his explanation. But the powers that be who define how computer math works have defined the type double in a way ... .that the values are not equal.



    The final upshot is that I don't need the precision of a double. If I case the values as singles, boom, works fine.



    I scratch my head. But it works.