Excel floating point representation - two 6s are not the same!

j66_europe

New Member
Joined
Jun 13, 2016
Messages
4
I'm really struggling when trying to understand how Excel deals with number representation. It would appear that (1.2 / 0.2) and (12 / 2) are NOT the same. Although both results appear as '6' in the formula bar, I noticed that in a pivot table, Excel 2013 was not summarising both values. Instead, the '6' appeared twice in the pivot results.

Bizarrely, the following formula returns true:
Code:
=IF((1.2/0.2)=(12/2),TRUE,FALSE)

However, clearly the two 6s can't be equivalent if the pivot table is not aggregating them.

I then experimented with dec2bin and dec2hex:
=DEC2BIN(1.2/0.2) yields 101 which is 5, yet...
=DEC2BIN(12/2) yields 110 which is 6

So what's going on here? How can I check how Excel is storing my numbers, because when it displays both as 6, and when my if function suggests the values are the same, how can they be? Is this expected behaviour? Thanks
 

Excel Facts

Do you hate GETPIVOTDATA?
Prevent GETPIVOTDATA. Select inside a PivotTable. In the Analyze tab of the ribbon, open the dropown next to Options and turn it off
Interesting. Not on my PC they aren't! In the immediate window:

Code:
? (1.2/0.2) = (12/2)
False
? INT(1.2/0.2) = INT(12/2)
False
? INT(1.2/0.2)
 5 
? INT(12/2)
 6 
? ROUND(1.2/0.2) = ROUND(12/2)
True

WBD
 
Upvote 0
Fascinating! Yes, in my VBA immediate window,

Code:
?INT(1.2/0.2)
 5

But in the spreadsheet, =INT(1.2/0.2) -> 6.

Can anyone explain what's going on? Thanks
 
Upvote 0

Forum statistics

Threads
1,215,218
Messages
6,123,676
Members
449,116
Latest member
HypnoFant

We've detected that you are using an adblocker.

We have a great community of people providing Excel help here, but the hosting costs are enormous. You can help keep this site running by allowing ads on MrExcel.com.
Allow Ads at MrExcel

Which adblocker are you using?

Disable AdBlock

Follow these easy steps to disable AdBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the icon in the browser’s toolbar.
2)Click on the "Pause on this site" option.
Go back

Disable AdBlock Plus

Follow these easy steps to disable AdBlock Plus

1)Click on the icon in the browser’s toolbar.
2)Click on the toggle to disable it for "mrexcel.com".
Go back

Disable uBlock Origin

Follow these easy steps to disable uBlock Origin

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back

Disable uBlock

Follow these easy steps to disable uBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back
Back
Top