Standard deviation returning wrong results

Salmonredd

New Member
Joined
Jan 24, 2005
Messages
22
When using stdev with three identicle values it should return 0 as a result. When the values 1.2, 1.2, 1.2 are used I am getting a result of .000000021...... Does anyone know why this is happening? It does it again with (3) 1.4's. I am considering trying to write a formula that will return 0 when the three values are identical, but doesn't that defeat the purpose of the stdev formula? Any help with this will be greatly appreciated.

EX: =IF(S44>2,STDEV(S20:S39),"")
 

Excel Facts

Select all contiguous cells
Pressing Ctrl+* (asterisk) will select the "current region" - all contiguous cells in all directions.
Salmonredd said:
When using stdev with three identicle values it should return 0 as a result. When the values 1.2, 1.2, 1.2 are used I am getting a result of .000000021...... Does anyone know why this is happening? It does it again with (3) 1.4's. I am considering trying to write a formula that will return 0 when the three values are identical, but doesn't that defeat the purpose of the stdev formula? Any help with this will be greatly appreciated.

EX: =IF(S44>2,STDEV(S20:S39),"")

If S20:S39 is numeric and without any blanks and

[1]

=SUM(1/(COUNTIF(S20:S39,S20:S39))

which is confirmed with control+shift+enter (instead of the usual enter), results in 1...

[2]

=STDEV(S20:S39)

would result in 0.

The point is that what you see in a cell is not necessarily what Excel sees.

If S20:S39 houses formulas, the numerical results they bring about has to be mapped onto the binary representation of numbers. At this point we get into the issue of floating point arithmetics on binary devices. I think you can find a lot on the latter thru Google.
 
Upvote 0

Forum statistics

Threads
1,214,905
Messages
6,122,172
Members
449,071
Latest member
cdnMech

We've detected that you are using an adblocker.

We have a great community of people providing Excel help here, but the hosting costs are enormous. You can help keep this site running by allowing ads on MrExcel.com.
Allow Ads at MrExcel

Which adblocker are you using?

Disable AdBlock

Follow these easy steps to disable AdBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the icon in the browser’s toolbar.
2)Click on the "Pause on this site" option.
Go back

Disable AdBlock Plus

Follow these easy steps to disable AdBlock Plus

1)Click on the icon in the browser’s toolbar.
2)Click on the toggle to disable it for "mrexcel.com".
Go back

Disable uBlock Origin

Follow these easy steps to disable uBlock Origin

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back

Disable uBlock

Follow these easy steps to disable uBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back
Back
Top