Thursday, March 01, 2007

Accurate Floating Point Addition

Will G, a friendly university tutor, asks "Does changing the order from largest to smallest first, among an infinite set of numbers following the pattern 1, 1/2, 1/3, etc, ever change the result's level of accuracy?" The sum is certainly the same no matter which way it's added up in real life because no matter how small a number is, it can always be represented by a fraction. However, a computer's floating point arithmetic doesn't use fractions. This creates the possibility of a loss in precision when adding an insignificant number to a relatively significant number. The set of numbers mentioned earlier can be added using the pattern 1 + 1/2 + 1/3 and so on. The running sum becomes exceedingly more significant relative to the next number being added. Eventually, the next number being added will become so insignificant relative to the running sum that it experiences a loss in precision! Consider adding the smallest numbers before the larger. The running sum becomes increasingly significant, once again, but so does the next number being added. Sure there will be a loss in precision, but it will be much less than adding the largest numbers together first. The order that a diverse set of numbers is added on a computer makes a difference, indeed!

The best way to add a diverse set of numbers together is by sorting them by magnitude, beginning with the smallest. Then proceed to add the numbers together from the front of the list. Once again, this ensures that as little precision as possible is sacrificed by the computer for the sake of using the floating point method of arithmetic.

No comments: