Weird issues with decimals

billyshears

Board Regular
Joined
Aug 29, 2013
Messages
61
I am using a couple sum if formulas in my spreadsheet. I have 2 cell totals that are supposed to equal each other and when they dont, they turn red.

It keeps coming up .00000000042 short of each other and I cant figure out why.

I am not multiplying anything, just adding based on certain categories.

Thanks in advance
 

Excel Facts

Shade all formula cells
To shade all formula cells: Home, Find & Select, Formulas to select all formulas. Then apply a light fill color.
Your computer stores numbers in binary floating point format with finite precision, and most decimals (like 0.1) are repeating decimals in binary the same way that 1/3 is in base 10. You should compare with a tolerance:

=if(abs(a2-a1) > 0.00001, this, that)
 
Last edited:
Upvote 0
I am using a couple sum if formulas in my spreadsheet. I have 2 cell totals that are supposed to equal each other and when they dont, they turn red. It keeps coming up .00000000042 short of each other and I cant figure out why. I am not multiplying anything, just adding based on certain categories.

It would be prudent to post a concrete example. Show 15 significant digits for numbers. Show formulas.

It is not so unusual. For example, =10.12-10-0.12 displays a negative infinitesimal value if the cell is formatted as General.

In general, the remedy is to explicitly round to the precision that you expect to be accurate. For example, =ROUND(10.12-10-0.12,2) returns exactly zero.
 
Upvote 0

Forum statistics

Threads
1,214,987
Messages
6,122,614
Members
449,090
Latest member
vivek chauhan

We've detected that you are using an adblocker.

We have a great community of people providing Excel help here, but the hosting costs are enormous. You can help keep this site running by allowing ads on MrExcel.com.
Allow Ads at MrExcel

Which adblocker are you using?

Disable AdBlock

Follow these easy steps to disable AdBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the icon in the browser’s toolbar.
2)Click on the "Pause on this site" option.
Go back

Disable AdBlock Plus

Follow these easy steps to disable AdBlock Plus

1)Click on the icon in the browser’s toolbar.
2)Click on the toggle to disable it for "mrexcel.com".
Go back

Disable uBlock Origin

Follow these easy steps to disable uBlock Origin

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back

Disable uBlock

Follow these easy steps to disable uBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back
Back
Top