Excel floating point representation - two 6s are not the same!

j66_europe

New Member
Joined
Jun 13, 2016
Messages
4
I'm really struggling when trying to understand how Excel deals with number representation. It would appear that (1.2 / 0.2) and (12 / 2) are NOT the same. Although both results appear as '6' in the formula bar, I noticed that in a pivot table, Excel 2013 was not summarising both values. Instead, the '6' appeared twice in the pivot results.

Bizarrely, the following formula returns true:
Code:
=IF((1.2/0.2)=(12/2),TRUE,FALSE)

However, clearly the two 6s can't be equivalent if the pivot table is not aggregating them.

I then experimented with dec2bin and dec2hex:
=DEC2BIN(1.2/0.2) yields 101 which is 5, yet...
=DEC2BIN(12/2) yields 110 which is 6

So what's going on here? How can I check how Excel is storing my numbers, because when it displays both as 6, and when my if function suggests the values are the same, how can they be? Is this expected behaviour? Thanks
 

Excel Facts

How to create a cell-sized chart?
Tiny charts, called Sparklines, were added to Excel 2010. Look for Sparklines on the Insert tab.
Interesting. Not on my PC they aren't! In the immediate window:

Code:
? (1.2/0.2) = (12/2)
False
? INT(1.2/0.2) = INT(12/2)
False
? INT(1.2/0.2)
 5 
? INT(12/2)
 6 
? ROUND(1.2/0.2) = ROUND(12/2)
True

WBD
 
Upvote 0
Fascinating! Yes, in my VBA immediate window,

Code:
?INT(1.2/0.2)
 5

But in the spreadsheet, =INT(1.2/0.2) -> 6.

Can anyone explain what's going on? Thanks
 
Upvote 0

Forum statistics

Threads
1,216,027
Messages
6,128,377
Members
449,445
Latest member
JJFabEngineering

We've detected that you are using an adblocker.

We have a great community of people providing Excel help here, but the hosting costs are enormous. You can help keep this site running by allowing ads on MrExcel.com.
Allow Ads at MrExcel

Which adblocker are you using?

Disable AdBlock

Follow these easy steps to disable AdBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the icon in the browser’s toolbar.
2)Click on the "Pause on this site" option.
Go back

Disable AdBlock Plus

Follow these easy steps to disable AdBlock Plus

1)Click on the icon in the browser’s toolbar.
2)Click on the toggle to disable it for "mrexcel.com".
Go back

Disable uBlock Origin

Follow these easy steps to disable uBlock Origin

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back

Disable uBlock

Follow these easy steps to disable uBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back
Back
Top