How To Work Around Floating-Point Accuracy/Comparison Problems

This article was previously published under Q69333
Retired KB Content Disclaimer
This article was written about products for which Microsoft no longer offers support. Therefore, this article is offered "as is" and will no longer be updated.
SUMMARY
To reliably test whether two floating-point variables or expressionsare equal (using IEEE format or MBF), you must subtract the twovariables being compared and test whether their difference is lessthan a value chosen at the limits of significance for single or doubleprecision. NO OTHER TEST FOR EQUALITY WILL BE RELIABLE. The followingformulas reliably test whether X and Y are equal:

  1. For single precision, you must test whether the difference of X and Y is less than the value 7 significant digits smaller than X or Y. Divide X or Y by 10^7 to find the comparison value. For example:
          ' To try this example in VBDOS.EXE:      ' 1. From the File menu, choose New Project.      ' 2. Copy the code example to the Code window.      ' 3. Press F5 to run the program.      IF ABS(X! - Y!) <= (X! / 10^7) THEN         PRINT "Equal within 7 digits"      ENDIF						
  2. For double precision, you must test whether the difference of X and Y is less than the value 15 significant digits smaller than X or Y. Divide X or Y by 10^15 to find the comparison value. For example:
          ' To try this example in VBDOS.EXE:      ' 1. From the File menu, choose New Project.      ' 2. Copy the code example to the Code window.      ' 3. Press F5 to run the program.      IF ABS(X# - Y#) <= (X# / 10^15) THEN         PRINT "Equal within 15 digits"      ENDIF						
The IEEE floating-point format is found in the Standard andProfessional Editions of Microsoft Visual Basic for MS-DOS, version1.0; in Microsoft QuickBasic for MS-DOS (QB87.EXE coprocessor versiononly), versions 3.0, 4.0, 4.0b, and 4.5; in Microsoft Basic Compiler forMS-DOS and MS OS/2, versions 6.0 and 6.0b; and in Microsoft BasicProfessional Development System (PDS) for MS-DOS and MS OS/2, versions7.0 and 7.1.

MBF (Microsoft Binary Format) is found in Microsoft QuickBasic forMS-DOS (QB.EXE non-coprocessor version only), versions 1.0, 1.01, 2.0,2.01, and 3.0, and in Microsoft GW-Basic Interpreter for MS-DOS, versions3.2, 3.22, and 3.23.

The information in this article is also included in the Help fileprovided with the Standard and Professional Editions of MicrosoftVisual Basic for MS-DOS, version 1.0.
MORE INFORMATION
NOTE: Significant digits in a calculated number can be lost due to the following: multiple calculations, especially addition of numbers far apart in value, or subtraction of numbers similar in value. When a number results from multiple calculations, you may need to change your test for equality to use fewer significant digits to reflect the mathematical loss of significant digits. If your test of significance uses too many significant digits, you may fail to discover that numbers compared for equality are actually equal within the possible limit of accuracy.

In the Basic products listed above that use IEEE floating-point format,intermediate calculations are performed in an internal 64-bittemporary register, which has more bits of accuracy than are stored insingle-precision or double-precision variables. This often results inan IF statement returning an error which states that the intermediatecalculation is not equal to the expression being compared. For example:
   X = 25   Y = 60.1   IF 1502.5 = (X * Y) THEN PRINT "equal"				
Running the above code will NOT print "equal". In contrast, thefollowing method using a placeholder variable will print "equal", butis still NOT a reliable technique as a test for equality:
   Z = 25 * 60.1   IF 1502.5 = Z THEN PRINT "equal"				
Note that explicit numeric type casts (! for single precision, # fordouble precision) will affect the precision in which calculations arestored and printed. Whichever type casting you perform, you may stillsee unexpected rounding results:
   PRINT 69.82! + 1    ' Single precision, prints 70.82.   PRINT 69.82# + 1    ' Double precision, prints 70.81999999999999.				
For an exact decimal (base 10) numeric representation, such as forcalculations of dollars and cents, you should use the CURRENCY (@)data type found in Visual Basic for MS-DOS, version 1.0 and Basic PDSfor MS-DOS, versions 7.0 and 7.1. The CURRENCY data type exactlystores up to 19 digits, with 4 digits after the decimal place.

Reference:

Both the IEEE and MBF standards attempt to balance accuracy andprecision with numeric range and speed. Accuracy measures how manysignificant bits of precision are not lost in calculations. Precisionrefers to the number of bits in the mantissa, which determines howmany decimal digits can be represented.

Both IEEE format and MBF store numbers of the form 1.x to the power ofy (where x and y are base 2 numbers; x is the mantissa, and y is theexponent).

MBF single precision has 24 bits of mantissa, and double precision has56 bits of mantissa. All MBF calculations are performed within just 24or 56 bits.

IEEE single precision has 24 bits of mantissa, and double precisionhas 53 bits of mantissa. However, all single-precision anddouble-precision IEEE calculations in Visual Basic for MS-DOS,version 1.0; in QuickBasic for MS-DOS, versions 3.0/4.x; in BasicCompiler for MS-DOS, versions 6.0, and 6.0b; and in Basic PDS forMS-DOS, versions 7.0 and 7.1re performed in a 64-bit temporary registerfor greater accuracy. As a result, IEEE calculations are more accuratethan MBF calculations, despite MBF's ability to represent more bits indouble precision.

Most numbers in decimal (base 10) notation do NOT have an exactrepresentation in the binary (base 2) floating-point storage formatused in single-precision and double-precision data types. Both IEEEformat and MBF cannot exactly represent (and must round off) allnumbers that are not of the form 1.x to the power of y (where x and yare base 2 numbers). The numbers that can be exactly represented arespread out over a very wide range. A high density of representablenumbers is near 1.0 and -1.0, but fewer and fewer representable numbersoccur as the numbers go towards 0 or infinity.

The above limitations often cause Basic to return floating-pointresults different than you might expect. More information on thistopic can be found in the Microsoft Knowledge Base by querying on thefollowing words:
floating and point and format and QuickBasic
The IEEE floating point format is documented in the following manuals:

  1. Pages 16-21 of "Microsoft QuickBasic 4.0: Basic Language Reference" for QuickBasic for MS-DOS, versions 4.0 and 4.0b. Note that the 4.0 manuals need a correction on page 19 to the MHex$ code example, as explained in a separate article found in the Microsoft Knowledge Base by querying on the following words:
    MHex$ AND 19
  2. Pages 12-17 of "Microsoft QuickBasic 4.5: Basic Language Reference" for QuickBasic for MS-DOS, version 4.5. Note that this optional manual must be ordered separately using an order card provided with version 4.5.
  3. Pages 702-705 of the "Microsoft Basic 7.0: Programmer's Guide" for Microsoft Basic PDS for MS-DOS, versions 7.0 and 7.1.
The MBF floating point format is documented in a separate articlewhich can be found in the Microsoft Knowledge Base by querying onthe following words:
MBF AND conversion AND exponent
VBmsdos QuickBas BasicCom 1.00 1.01 2.00 2.01 3.00 3.20 3.22 3.23 4.00 4.00b 4.50 6.00 6.00b 7.00 7.10
Properties

Article ID: 69333 - Last Review: 08/16/2005 21:27:24 - Revision: 2.2

Microsoft Visual Basic for MS-DOS, Microsoft QuickBasic 4.0, Microsoft QuickBASIC 4.0b, Microsoft QuickBasic 4.5 for MS-DOS, Microsoft BASIC Compiler 6.0, Microsoft BASIC Compiler 6.0b, Microsoft BASIC Professional Development System 7.0, Microsoft BASIC Professional Development System 7.1, Microsoft GW-BASIC 3.2, Microsoft GW-BASIC 3.22, Microsoft GW-BASIC 3.23

  • kbhowto KB69333
Feedback