Weird issues with decimals

billyshears

Board Regular
Joined
Aug 29, 2013
Messages
58
I am using a couple sum if formulas in my spreadsheet. I have 2 cell totals that are supposed to equal each other and when they dont, they turn red.

It keeps coming up .00000000042 short of each other and I cant figure out why.

I am not multiplying anything, just adding based on certain categories.

Thanks in advance
 

Excel Facts

Easy bullets in Excel
If you have a numeric keypad, press Alt+7 on numeric keypad to type a bullet in Excel.

shg

MrExcel MVP
Joined
May 7, 2008
Messages
21,785
Office Version
  1. 2010
Platform
  1. Windows
Your computer stores numbers in binary floating point format with finite precision, and most decimals (like 0.1) are repeating decimals in binary the same way that 1/3 is in base 10. You should compare with a tolerance:

=if(abs(a2-a1) > 0.00001, this, that)
 
Last edited:

joeu2004

Well-known Member
Joined
Mar 2, 2014
Messages
3,028
Office Version
  1. 2010
Platform
  1. Windows
I am using a couple sum if formulas in my spreadsheet. I have 2 cell totals that are supposed to equal each other and when they dont, they turn red. It keeps coming up .00000000042 short of each other and I cant figure out why. I am not multiplying anything, just adding based on certain categories.

It would be prudent to post a concrete example. Show 15 significant digits for numbers. Show formulas.

It is not so unusual. For example, =10.12-10-0.12 displays a negative infinitesimal value if the cell is formatted as General.

In general, the remedy is to explicitly round to the precision that you expect to be accurate. For example, =ROUND(10.12-10-0.12,2) returns exactly zero.
 

Watch MrExcel Video

Forum statistics

Threads
1,130,045
Messages
5,639,750
Members
417,108
Latest member
Thein Than

We've detected that you are using an adblocker.

We have a great community of people providing Excel help here, but the hosting costs are enormous. You can help keep this site running by allowing ads on MrExcel.com.
Allow Ads at MrExcel

Which adblocker are you using?

Disable AdBlock

Follow these easy steps to disable AdBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the icon in the browser’s toolbar.
2)Click on the "Pause on this site" option.
Go back

Disable AdBlock Plus

Follow these easy steps to disable AdBlock Plus

1)Click on the icon in the browser’s toolbar.
2)Click on the toggle to disable it for "mrexcel.com".
Go back

Disable uBlock Origin

Follow these easy steps to disable uBlock Origin

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back

Disable uBlock

Follow these easy steps to disable uBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back
Top