Decimal precision in VBA macro

FredM400

New Member
Joined
Jul 22, 2008
Messages
26
I need some help with number precision and Variant data types.

vIdx in defined as Integer and used as a array index

I have 3 variables, all defined as Variants:
vBaseCellValue(vIdx)
vCompCellValue(vIdx)
vCellVariance

I have the statement;

vCellVariance = vBaseCellValue(vIdx) - vCompCellValue(vIdx)

Under the conditions
vBaseCellValue(vIdx) = 29435.15
vCompCellValue(vIdx) = 29435.15

I would expect the variable vCellVariance to be equal to zero.

The debug Immediate window say the variable vCellVariance is actual equal to 7.275958E-12.
A really really small number, but not zero.

My compare to zero fails and the program goes off down the wrong path.


How can I the correct answer of Zero????
 

Excel Facts

Add Bullets to Range
Select range. Press Ctrl+1. On Number tab, choose Custom. Type Alt+7 then space then @ sign (using 7 on numeric keypad)
A brute force way would be to use:

vCellVariance = ROUND(vBaseCellValue(vIdx) - vCompCellValue(vIdx),2)
 
Upvote 0

Forum statistics

Threads
1,224,534
Messages
6,179,391
Members
452,909
Latest member
VickiS

We've detected that you are using an adblocker.

We have a great community of people providing Excel help here, but the hosting costs are enormous. You can help keep this site running by allowing ads on MrExcel.com.
Allow Ads at MrExcel

Which adblocker are you using?

Disable AdBlock

Follow these easy steps to disable AdBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the icon in the browser’s toolbar.
2)Click on the "Pause on this site" option.
Go back

Disable AdBlock Plus

Follow these easy steps to disable AdBlock Plus

1)Click on the icon in the browser’s toolbar.
2)Click on the toggle to disable it for "mrexcel.com".
Go back

Disable uBlock Origin

Follow these easy steps to disable uBlock Origin

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back

Disable uBlock

Follow these easy steps to disable uBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back
Back
Top