Best solution for Big Dataset search and then making "Statement of Account sheet" for each record

Meesam_ali

Board Regular
Joined
Jul 10, 2016
Messages
106
Hello Everybody,
I am myself an excel VBA programmer as well as programmer in other languages i.e. Java, JavaScript, Python and C++. Currently I got a project where I have a huge dataset (more than 20,000 records spread over 25 columns) and not only that I have to search each record, I have to actually make a "Statement of Account" for each UNIQUE record. I already have made the full code and it works fine but my problem is that for such a huge data-set its super super slow. This is for obvious reasons as the code has to not only search through entire record but then create a NEW statement of account sheet for each record (which is quite CPU intensive task) and not something excel is good to handle. Can any expert person recommend me any alternate solution using SQL databases (I can shift my data to SQL) and then using some other software i.e. Python or Java to make the final result sheets or layouts. Excel is good for many things but i believe when data gets bigger than certain size then Excel isn't the most suitable solution and Databases should be the right solution.
Any suggestions / ideas are welcome.
 

Some videos you may like

Excel Facts

Move date out one month or year
Use =EDATE(A2,1) for one month later. Use EDATE(A2,12) for one year later.

sandy666

Well-known Member
Joined
Oct 24, 2015
Messages
6,688
maybe PowerBi.
I can't say more because I don't like PBI but you can try
 

Meesam_ali

Board Regular
Joined
Jul 10, 2016
Messages
106
Since then I've been looking into PowerBI. Only issue being no support of VBA up till now. But hopefully its gonna help me reduce the search time considerably.
 

Watch MrExcel Video

Forum statistics

Threads
1,108,652
Messages
5,524,096
Members
409,559
Latest member
rjoslin414

This Week's Hot Topics

Top