wildwilly1111
New Member
- Joined
- Mar 16, 2011
- Messages
- 1
In this sample, test values that round to -0.102 or -0.104 lookup incorrectly, returning the next higher result in the table. Test values that round to the the other values in the lookup range return correct results (e.g. -0.100999, -0.105000). Truncating the rounded value causes all lookups to work, even though the rounded and truncated values test to equality.
Can someone help me understand why the rounded loookups sometimes fail and justify why the truncation step (seemingly a no-op) is the correct resolution?
Excel 2007
Can someone help me understand why the rounded loookups sometimes fail and justify why the truncation step (seemingly a no-op) is the correct resolution?
Excel Workbook | |||||
---|---|---|---|---|---|
A | B | C | |||
1 | Test value | -0.103702 | |||
2 | Rounded | -0.104 | |||
3 | Rounded+Truncated | -0.104 | |||
4 | Rounded = Rounded+Truncated? | Yes | |||
5 | Lookup [Rounded] | F | |||
6 | Lookup [Rounded+Truncated] | E | |||
7 | Lookups Equal? | No | |||
8 | |||||
9 | Lookup Table | -0.110 | H | ||
10 | -0.108 | G | |||
11 | -0.106 | F | |||
12 | -0.104 | E | |||
13 | -0.102 | D | |||
14 | -0.100 | C | |||
15 | -0.098 | B | |||
16 | -0.096 | A | |||
Sheet1 |
Cell Formulas | ||
---|---|---|
Range | Formula | |
B2 | =MROUND(B1,-0.002) | |
B3 | =TRUNC(B2,3) | |
B4 | =IF(B2=B3,"Yes","No") | |
B5 | =LOOKUP(B2,B9:B16,C9:C16) | |
B6 | =LOOKUP(B3,B9:B16,C9:C16) | |
B7 | =IF(B5=B6,"Yes","No") |
Last edited: