FIX: Result Differs When Comparing Single w/ Double Precision

The information in this article applies to:

- The Standard and Professional Editions of Microsoft Visual Basic for

  Windows, version 2.0

SYMPTOMS

When you compare a real number stored as a Single precision variable to the same real number stored as a Double precision variable, the result may be that they are not equal. Storage of real numbers is different within the two data types. Therefore, the number may be represented differently, so a check for equivalence can return false.

WORKAROUND

In Microsoft Visual Basic version 2.0 for Windows, this problem occurs only on computers that do not have coprocessors. If your computer does not have a coprocessor, add some extra code when comparing data stored in Single data types to those stored in Double data types.

STATUS

Microsoft has confirmed this to be a problem in the Standard and Professional Editions of Microsoft Visual Basic version 2.0 for Windows. It was corrected in Microsoft Visual Basic version 3.0 for Windows.

MORE INFORMATION

Steps to Reproduce the Problem

1. Start VB.EXE.

2. Add the following in the Form_Click event procedure of Form1:

         Print 12.3! = 12.3#  '** Note the '#' sign disappears.
   

3. Press the F5 key to run the example, and click Form1. If the result is '0' then a coprocessor is not installed, if the result is '-1' then a coprocessor is installed or you are on a 486 with a built in coprocessor.