Lookup of rounded number fails for certain numbers

wildwilly1111

New Member
Joined
Mar 16, 2011
Messages
1
In this sample, test values that round to -0.102 or -0.104 lookup incorrectly, returning the next higher result in the table. Test values that round to the the other values in the lookup range return correct results (e.g. -0.100999, -0.105000). Truncating the rounded value causes all lookups to work, even though the rounded and truncated values test to equality.

Can someone help me understand why the rounded loookups sometimes fail and justify why the truncation step (seemingly a no-op) is the correct resolution?
Excel Workbook
ABC
1Test value-0.103702
2Rounded-0.104
3Rounded+Truncated-0.104
4Rounded = Rounded+Truncated?Yes
5Lookup [Rounded]F
6Lookup [Rounded+Truncated]E
7Lookups Equal?No
8
9Lookup Table-0.110H
10-0.108G
11-0.106F
12-0.104E
13-0.102D
14-0.100C
15-0.098B
16-0.096A
Sheet1
Excel 2007
Cell Formulas
RangeFormula
B2=MROUND(B1,-0.002)
B3=TRUNC(B2,3)
B4=IF(B2=B3,"Yes","No")
B5=LOOKUP(B2,B9:B16,C9:C16)
B6=LOOKUP(B3,B9:B16,C9:C16)
B7=IF(B5=B6,"Yes","No")
 
Last edited:

Excel Facts

Easy bullets in Excel
If you have a numeric keypad, press Alt+7 on numeric keypad to type a bullet in Excel.

Forum statistics

Threads
1,215,008
Messages
6,122,672
Members
449,091
Latest member
peppernaut

We've detected that you are using an adblocker.

We have a great community of people providing Excel help here, but the hosting costs are enormous. You can help keep this site running by allowing ads on MrExcel.com.
Allow Ads at MrExcel

Which adblocker are you using?

Disable AdBlock

Follow these easy steps to disable AdBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the icon in the browser’s toolbar.
2)Click on the "Pause on this site" option.
Go back

Disable AdBlock Plus

Follow these easy steps to disable AdBlock Plus

1)Click on the icon in the browser’s toolbar.
2)Click on the toggle to disable it for "mrexcel.com".
Go back

Disable uBlock Origin

Follow these easy steps to disable uBlock Origin

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back

Disable uBlock

Follow these easy steps to disable uBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back
Back
Top