SHAZAM Rounding Errors

Rounding Errors


Rounding means shortening the fractional part of the number. Computer programs have a limit to the amount of precision that numbers can be stored. SHAZAM is designed to retain the maximum number of digits possible in computer memory, even if they are not all printed on the output file. SHAZAM also uses highly accurate computer algorithms to minimize the impact of rounding errors.

Although rounding errors may appear to be a trivial problem, the cumulative effect of rounding errors can have a significant impact on numerical calculations. To illustrate this point, the following example has been created. In each successive row, one significant digit is dropped and the numbers are rounded. By comparing the products of the four numbers in each successive row, the effect of rounding errors can be seen.

      2/13        6/13        9/13       10/13        PRODUCT

    .1538462    .4615385    .6923077    .7692308    .037813819
    .153846     .461538     .692308     .769231     .037813755
    .15385      .46154      .69231      .76923      .037814962
    .1538       .4615       .6923       .7692       .037797376
    .154        .462        .692        .769        .037861266
    .15         .46         .69         .77         .036659700
    .2          .5          .7          .8          .056000000

Note the difference between the products in the first and last row. There is a 48 percentage change between the numbers. It should now be clear that rounding errors can significantly alter numerical results.

The Pentium Bug

Rounding error became the subject of much public discussion and controversy in 1994 when problems with the pentium chip were described in computer newsgroups. The pentium is the 64-bit microprocessor for the PC developed by Intel as the successor to the 386 and 486 microprocessors. Consider the following calculation:

      ( (4195835 / 3145727) * 3145727) - 4195835

The result should be 0. But computers with the faulty pentium chip get the result -256. This was recognized as a bug. After much embarrassing publicity Intel corrected the problem and agreed to supply users with replacement chips.

SHAZAM can detect the presence of the pentium bug with the CHECKOUT command.


Home [SHAZAM Guide home]