The Naked Scientists

The Naked Scientists Forum

Author Topic: What's going on with the Windows calculator?  (Read 2631 times)

Offline techmind

  • Hero Member
  • *****
  • Posts: 934
  • Un-obfuscated
    • View Profile
What's going on with the Windows calculator?
« on: 28/05/2013 23:46:17 »
I just noticed some weirdness with the Windows calculator.
If I enter the following numbers, then press [1/x]   (that's the reciprocal button) I get:
5666.0   -> 1.7649135192375573596893752206142e-4
5666.1   -> 0.0001764882370590000176488237059
5666.2   -> 1.7648512230418975680350146482652e-4
5666.3   -> 1.7648200765931913241445034678715e-4
5666.4   -> 1.7647889312438232387406466186644e-4
5666.5   -> 0.000176475778699373510985617224036
5666.6   -> 1.7647266438428687396322309674231e-4
5666.7   -> 1.7646955017911659343180334233328e-4
5666.8   -> 1.7646643608385685042704877532293e-4
5666.9   -> 0.0001764633220985018263953837194939
5667.0   -> 1.7646020822304570319392976883713e-4

That's odd. Why do certain numbers not appear in scientific (e-4) notation?

Is there a bigger pattern to this?
Is this purely a display-bug, or is this the tip of a bigger iceberg?

Do you see the same? On what Windows versions?

I'm on Windows XP (sp3, on an old Pentium4 processor)

It doesn't seem to matter whether the calculator is in the "scientific" or "normal" mode.
Working with the same numbers, but x10 bigger (i.e. 56660 ... 56670) and again taking the reciprocal gives similar, yet subtly different results again!  :o
« Last Edit: 28/05/2013 23:56:38 by techmind »


Offline CliffordK

  • Neilep Level Member
  • ******
  • Posts: 6321
  • Thanked: 3 times
  • Site Moderator
    • View Profile
Re: What's going on with the Windows calculator?
« Reply #1 on: 29/05/2013 00:39:02 »
You're getting a mix of exponential (scientific) and decimal notation.

When I did the calculations on my computer for a few key numbers, using SpeedCrunch.

1/5666.1 = 0.00017648823705900001764882370590000 176488237059000
1/5666.5 = 0.00017647577869937351098561722403600 105885467219624
1/5666.9 = 0.00017646332209850182639538371949390 319222149676190
1/5666.0 = 0.00017649135192375573596893752206141 899046946699612
1/5666.2 = 0.00017648512230418975680350146482651 512477498146906

Note that the digit following your displayed number is a ZERO.

When Windows rounds the number, it apparently is getting a whole number, and thus chooses to display it in decimal form.  On the other numbers, the digit following the last displayed digit is non-zero, and thus it is choosing scientific notation.

Actually, when I put it into courier font, it is clear that you loose a couple of digits on the trailing end due to the zeros.  I.E.  It isn't just the digits following the last displayed digit, but rather if the last displayed digit is full precision, then it is written in scientific notation.  If it is not, then it would be written in decimal notation.

I presume it also has to do with the number of leading zeros that you have (4, including the one before the decimal).  (e-4) takes 3 spaces.  I think this means that you can actually display one extra digit of precision in scientific notation than you can display in decimal notation.

For the three numbers displayed, that extra digit of precision is irrelevant.

If you had 3 leading zeros (10/5666.0) the precision in decimal and scientific notation would be the same, so I would predict that they all would be displayed in decimal notation.

If you had 5 leading zeros (0.1/5666.1, 0.1//5666.5, & 0.1//5666.9), at least the first one will be in decimal notation.  Probably the second, but definitely not the third.  At least that would be my guess.

The Naked Scientists Forum

Re: What's going on with the Windows calculator?
« Reply #1 on: 29/05/2013 00:39:02 »


SMF 2.0.10 | SMF © 2015, Simple Machines
SMFAds for Free Forums