floating point error

  1. W

    Accuracy issue (presumably) with low numbers

    My colleague has an issue with summing a list. I've shown the original below, and the permutations I've tried to the right which all result in the correct zero sum. Any idea how this happens? <colgroup><col><col span="4"></colgroup><tbody> Cell Original A>Z Z>A Swapped A1 22.05 -427.85 661...
  2. C

    Precision IEEE sum to 0 error

    I have this spreadsheet with 5 months of data. For reasons unknown to me, 3 months have data that add up to 0 and the remaining 2 months are 0's. When summing the 3 numbers, Excel is smart enough to account for the floating-point issues and it returns 0. However, if you include the two 0's in...
  3. L

    Series Issue Within Charts Once Links Are Broken

    I have posted this on the Home - Microsoft Community site but I thought I would post this other places just to see if anyone is familiar with this issue.<o:p></o:p> <o:p> </o:p> I have a very peculiar issue with series in charts that have been converted to values once links are broken. I cannot...
  4. D

    Error with int() calculation

    Hi All, This is driving me batty! The int() function returns obviously wrong results and I can't figure out why. I've simplified things with the following bit of code: Sub Int_Error() x = 39.8 y = x * 100 z = Int(y) End Sub If you step through this and watch the values , 'z'...
  5. Greg Truby

    Floating Point Error in For/Next Loop Counter

    Well, I'm glad this just happed to occur in something where it wasn't mission critical... Try this: Sub foo() For p = -1 To 1 Step 0.2 With Cells(1, (p + 1) * 5 + 1) .NumberFormat = "0%_);[Red](0%);0%" .Value = p End With Next p End Sub What do...

Some videos you may like

This Week's Hot Topics

Top