Maximum Size for a 2 dimensional array

Shaft120

New Member
Joined
Sep 24, 2009
Messages
14
Hi - Been a while, but I'm hoping to find a way to fix a limitation on a reconciliation tool I designed a while back to compare two extracts from an SQL database.

In particular I'm having trouble on memory limitations when Resizing a Variant array:

ReDim Ukey(1 To MyDict.Count, 1 To ((H - 1) * 3) + 2)

This equates to a Redim of Ukey (1 To 221, 1 To 58589)

Which is c13m values. I thought there was a limitation of about 25m, but I cant recall why I thought that. Anyway, any thoughts on a workaround?

Context is that I have a primary key stored in MyDict and Each files values in a separate Array. I resize this 3rd array to combine both and add a comparison column as well, then parse the data in through loops (tried using Dictionary print previously, but that is even more limited in size, so had to rewrite it).

Thoughts appreciated.
 

Excel Facts

What is the shortcut key for Format Selection?
Ctrl+1 (the number one) will open the Format dialog for whatever is selected.
The code below works for me. You can always use worksheet cells with screen updating disabled, but it may be slower.
Could you do it all within SQL?

Code:
Sub Macro1()
Dim ukey, i, j, maxx, maxy
maxx = 1000#
maxy = 100000#
ReDim ukey(1 To maxx, 1 To maxy)
For i = 1 To maxx
    For j = 1 To maxy
        ukey(i, j) = i + j
Next j, i
MsgBox ukey(maxx, maxy)
End Sub
 
Upvote 0
If you are using a dictionary why not store the values associated with each key in the dictionary in separate arrays?
 
Upvote 0
Hi Norie. I did originaly design it that way, but dictionary is even more limited in memory printing back to the sheet at the end. In order to keep the maximum possible dataset, i use 2 sourcr arrays and then the dictionary just for primary keys to loop through for its calculations as I populate the final array. Its the last one which was causing the issue, but once populated it's easier to print it into the sheet.

Anyway - I've solved the problem with a workaround. If I declare the array as String, it removes the memory problem. It means I have to know which columns of data to treat as values - so the user has to define these up front before running, but then when I am doing compares on a value column, I can use Val() for that data. Likewise at the end I can convert those columns I know have values in them after they have been put back into the worksheet.
 
Upvote 0
What problems were you actually having?

I know there's a limit to how much data you can write back to a sheet in one go but I'm not sure that's a memory issue.

When I've run into that problem I've used some novel approaches, seem to recall one time writing the data, as it was being processed, out to a CSV file and then using ADO to read that back in.
 
Upvote 0

Forum statistics

Threads
1,213,564
Messages
6,114,334
Members
448,567
Latest member
Kuldeep90

We've detected that you are using an adblocker.

We have a great community of people providing Excel help here, but the hosting costs are enormous. You can help keep this site running by allowing ads on MrExcel.com.
Allow Ads at MrExcel

Which adblocker are you using?

Disable AdBlock

Follow these easy steps to disable AdBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the icon in the browser’s toolbar.
2)Click on the "Pause on this site" option.
Go back

Disable AdBlock Plus

Follow these easy steps to disable AdBlock Plus

1)Click on the icon in the browser’s toolbar.
2)Click on the toggle to disable it for "mrexcel.com".
Go back

Disable uBlock Origin

Follow these easy steps to disable uBlock Origin

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back

Disable uBlock

Follow these easy steps to disable uBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back
Back
Top