You are currently offline, waiting for your internet to reconnect

Problems vith Statistical Functions and Large Numbers in Excel

Support for Office 2003 has ended

Microsoft ended support for Office 2003 on April 8, 2014. This change has affected your software updates and security options. Learn what this means for you and how to stay protected.

This article was previously published under Q158071
In Microsoft Excel, when you use any of the following statisticalfunctions in a formula:
The formula may return an incorrect result.
This behavior may occur when the formula refers to very large numbersthat, when summed, multiplied, or squared, exceed 15 digits in length.
Microsoft Excel supports a maximum of 15 significant digits at all times.This limit applies to a value that is calculated by a formula. Because ofthis limitation, if at any time a formula calculates a value that exceeds15 digits in length, digits beyond the fifteenth significant digit arechanged to zeroes. This may cause the formula to return an incorrectresult.

For example, when you use the STDEV function to calculate the standarddeviation of a set of numbers, part of the function sums the numbers andthen squares the result. If this number exceeds 15 digits in length,digits beyond the fifteenth digit are changed to zeroes. This affects thefinal result delivered by the formula. To see an example of this behavior,enter the following data in a worksheet:
   A1: 999999          B1: 99999999   A2: 1000000         B2: 100000000   A3: 1000001         B3: 100000001   A4: =STDEV(A1:A3)   B4: =STDEV(B1:B3)				
Although both formulas return the same standard deviation, 1, the secondformula returns a zero. Because the square of the sum of the three valuesin B1:B3 is greater than 15 digits in length, digits beyond the fifteenthdigit are changed to zeroes. This causes the formula to return anincorrect result.

This behavior may occur when you use any of the statistical functionslisted in this article and you work with very large values. This is truebecause these functions all use squared values, which makes it probablethat the limit of 15 significant digits is exceeded.

The exact formulas used by functions in Microsoft Excel are listed in theHelp topic for each function in Microsoft Excel Help.
For additional information about the number of significant digits that Excel retains, click the following article number to view the article in the Microsoft Knowledge Base:
65903 Number of significant digits MS Excel retains
XL2002 XL2000 XL98 XL97 5.00a 5.00c 8.00

Article ID: 158071 - Last Review: 06/12/2007 03:38:16 - Revision: 2.5

  • Microsoft Office Excel 2003
  • Microsoft Excel 2002 Standard Edition
  • Microsoft Excel 2000 Standard Edition
  • Microsoft Excel 97 Standard Edition
  • Microsoft Excel 95 Standard Edition
  • Microsoft Excel 95a
  • Microsoft Excel 5.0 Standard Edition
  • Microsoft Excel 5.0c
  • Microsoft Excel 2004 for Mac
  • Microsoft Excel 2001 for Mac
  • Microsoft Excel 98 for Macintosh
  • Microsoft Excel 5.0 for Macintosh
  • Microsoft Excel 5.0a for Macintosh
  • KB158071