Standard deviation returning wrong results

Salmonredd

New Member
Joined
Jan 24, 2005
Messages
22
When using stdev with three identicle values it should return 0 as a result. When the values 1.2, 1.2, 1.2 are used I am getting a result of .000000021...... Does anyone know why this is happening? It does it again with (3) 1.4's. I am considering trying to write a formula that will return 0 when the three values are identical, but doesn't that defeat the purpose of the stdev formula? Any help with this will be greatly appreciated.

EX: =IF(S44>2,STDEV(S20:S39),"")
 

Excel Facts

Which came first: VisiCalc or Lotus 1-2-3?
Dan Bricklin and Bob Frankston debuted VisiCalc in 1979 as a Visible Calculator. Lotus 1-2-3 debuted in the early 1980's, from Mitch Kapor.

Aladin Akyurek

MrExcel MVP
Joined
Feb 14, 2002
Messages
85,209
Salmonredd said:
When using stdev with three identicle values it should return 0 as a result. When the values 1.2, 1.2, 1.2 are used I am getting a result of .000000021...... Does anyone know why this is happening? It does it again with (3) 1.4's. I am considering trying to write a formula that will return 0 when the three values are identical, but doesn't that defeat the purpose of the stdev formula? Any help with this will be greatly appreciated.

EX: =IF(S44>2,STDEV(S20:S39),"")

If S20:S39 is numeric and without any blanks and

[1]

=SUM(1/(COUNTIF(S20:S39,S20:S39))

which is confirmed with control+shift+enter (instead of the usual enter), results in 1...

[2]

=STDEV(S20:S39)

would result in 0.

The point is that what you see in a cell is not necessarily what Excel sees.

If S20:S39 houses formulas, the numerical results they bring about has to be mapped onto the binary representation of numbers. At this point we get into the issue of floating point arithmetics on binary devices. I think you can find a lot on the latter thru Google.
 

Forum statistics

Threads
1,148,523
Messages
5,747,180
Members
424,068
Latest member
Salim khamis

We've detected that you are using an adblocker.

We have a great community of people providing Excel help here, but the hosting costs are enormous. You can help keep this site running by allowing ads on MrExcel.com.
Allow Ads at MrExcel

Which adblocker are you using?

Disable AdBlock

Follow these easy steps to disable AdBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the icon in the browser’s toolbar.
2)Click on the "Pause on this site" option.
Go back

Disable AdBlock Plus

Follow these easy steps to disable AdBlock Plus

1)Click on the icon in the browser’s toolbar.
2)Click on the toggle to disable it for "mrexcel.com".
Go back

Disable uBlock Origin

Follow these easy steps to disable uBlock Origin

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back

Disable uBlock

Follow these easy steps to disable uBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back
Top