VBA - text import puts contents all in one cell, need comma delimit instead

Sayre

Board Regular
Joined
Apr 12, 2005
Messages
180
Hi, I found and modified the below code and corresponding custom function hoping it will work faster than Workbooks.OpenText on large csv file imports (500K rows+ in some cases). It works great but writes all the data to just one cell in the worksheet which obviously won't work for large files.

How do I modify it so it writes it to the sheet more like a proper comma delimited across man rows/columns? I found threads on split function but not sure if that's the way to go or how to incorporate it if it is. Hoping there's a few simple lines I can add to the below script or custom function. If its more complicated than that could anyone at least point me towards a potential solution? Any hep truly appreciated....

Here's the script...
Code:
Sub TxtImporter2()
Dim f As String, flPath As String
Dim i As Long, j As Long
    Dim sPath As String
Dim ws As Worksheet
Application.DisplayAlerts = False
Application.ScreenUpdating = False
flPath = Sheets("START").Cells(9, 1).Value & Application.PathSeparator
i = ThisWorkbook.Worksheets.Count
j = Application.Workbooks.Count
f = Dir(flPath & "*.csv")
Do Until f = ""
Workbooks.Add
 Cells(1, 1) = LoadTextFile2(flPath & f)
    Workbooks(j + 1).Worksheets(1).Copy After:=ThisWorkbook.Worksheets(i)
    ThisWorkbook.Worksheets(i + 1).Name = Left(f, Len(f) - 4)
    Workbooks(j + 1).Close SaveChanges:=False
    i = i + 1
    f = Dir
Loop
Application.DisplayAlerts = True
End Sub


Here's the corresponding function...
Code:
 ' \\ Function to return the full content of a text file as a string
Public Function LoadTextFile2(sFile As String) As String
    Dim iFile As Integer
     
Dim lineSplit As Variant

    ' \\ Use FreeFile to supply a file number that is not already in use
    iFile = FreeFile
     ' \\ ' Open file for input.
    Open sFile For Input As #iFile


     
     ' \\ Return (Read) the whole content of the file to the function
    LoadTextFile2 = Input$(LOF(iFile), iFile)
    'Split(Symbols, ".")
     
    Close #iFile
    
End Function
 

Excel Facts

When did Power Query debut in Excel?
Although it was an add-in in Excel 2010 & Excel 2013, Power Query became a part of Excel in 2016, in Data, Get & Transform Data.
Can you give us an example of the data format?
 
Upvote 0
Yes absolutely. Here's a snippet of the largest csv file. This is actually 300+MB and although it comes to me saved with a .csv extension I now realize it is in space delimited format. Most of the smaller files really are comma delimited however so I guess it would be really great if the code could detect whether the file is space or comma delimited and then write it as such t the sheet either way.
But I'll crawl before I walk on that one ;)

I'm using code tags to get the below in here but its really just a text file:

Code:
HOSTNAME SERVER_ID LINE_ID CONTENT
USSJCIRCL001PRD 1065 __LINEID__ 3/9/2014	4:28:03 AM	4	0	1150	Microsoft Antimalware	N/A	USSJCIRCL001PRD.NG.CORP	%%860 4.3.0220.0  1.1.10302.0 1.167.1482.0 
USSJCIRCL001PRD 1065 __LINEID__ 3/9/2014	4:34:22 AM	4	0	7036	Service Control Manager	N/A	USSJCIRCL001PRD.NG.CORP	Windows Modules Installer running 
USSJCIRCL001PRD 1065 __LINEID__ 3/9/2014	4:37:01 AM	4	0	7036	Service Control Manager	N/A	USSJCIRCL001PRD.NG.CORP	WinHTTP Web Proxy Auto-Discovery Service running 
USSJCIRCL001PRD 1065 __LINEID__ 3/9/2014	4:47:17 AM	4	0	7040	Service Control Manager	NT AUTHORITY\SYSTEM		USSJCIRCL001PRD.NG.CORP	Windows Modules Installer demand start auto start TrustedInstaller 
USSJCIRCL001PRD 1065 __LINEID__ 3/9/2014	4:47:18 AM	4	0	7040	Service Control Manager	NT AUTHORITY\SYSTEM		USSJCIRCL001PRD.NG.CORP	Windows Modules Installer auto start demand start TrustedInstaller 
USSJCIRCL001PRD 1065 __LINEID__ 3/9/2014	4:47:18 AM	4	0	7036	Service Control Manager	N/A	USSJCIRCL001PRD.NG.CORP	Windows Modules Installer stopped 
USSJCIRCL001PRD 1065 __LINEID__ 3/9/2014	4:53:08 AM	4	0	7036	Service Control Manager	N/A	USSJCIRCL001PRD.NG.CORP	Windows Modules Installer running 
USSJCIRCL001PRD 1065 __LINEID__ 3/9/2014	4:53:31 AM	4	0	7036	Service Control Manager	N/A	USSJCIRCL001PRD.NG.CORP	WinHTTP Web Proxy Auto-Discovery Service stopped 
USSJCIRCL001PRD 1065 __LINEID__ 3/9/2014	5:04:49 AM	4	0	7040	Service Control Manager	NT AUTHORITY\SYSTEM		USSJCIRCL001PRD.NG.CORP	Windows Modules Installer demand start auto start TrustedInstaller 
USSJCIRCL001PRD 1065 __LINEID__ 3/9/2014	5:04:49 AM	4	0	7040	Service Control Manager	NT AUTHORITY\SYSTEM		USSJCIRCL001PRD.NG.CORP	Windows Modules Installer auto start demand start TrustedInstaller 
USSJCIRCL001PRD 1065 __LINEID__ 3/9/2014	5:04:50 AM	4	0	7036	Service Control Manager	N/A	USSJCIRCL001PRD.NG.CORP	Windows Modules Installer stopped 
USSJCIRCL001PRD 1065 __LINEID__ 3/9/2014	5:12:00 AM	4	0	7036	Service Control Manager	N/A	USSJCIRCL001PRD.NG.CORP	Windows Modules Installer running
 
Upvote 0
... and here's an example of one of the .csv's that is actually comma separated:

Code:
hostname,hostuid,lineid,Disk_###,Status,Size_GB,Free_GB,Dyn,Gpt
USSJCIRCL001PRD,1065,__LINEID__,Disk_0, Online, 279.0, 0.0, , 
USSJCIRCL002PRD,1066,__LINEID__,Disk_0, Online, 279.0, 0.0, , 
USSJCIRCL003PRD,1067,__LINEID__,Disk_0, Online, 279.0, 0.0, , 
USSJCIRCL004PRD,1068,__LINEID__,Disk_0, Online, 279.0, 0.0, , 
USSJCIRDB001PRD,1056,__LINEID__,Disk_0, Online, 136.0, 0.0, , 
USSJCIRDB001PRD,1056,__LINEID__,Disk_1, Online, 186.0, 0.0, , 
USSJCIRDB001PRD,1056,__LINEID__,Disk_2, Online, 1000.0, 0.0, , 
USSJCIRDB001PRD,1056,__LINEID__,Disk_3, Online, 1000.0, 0.0, , 
USSJCIRDB001PRD,1056,__LINEID__,Disk_4, Online, 1000.0, 0.0, , 
USSJCIRDB001PRD,1056,__LINEID__,Disk_5, Online, 1250.0, 0.0, , 
USSJCIRDB001PRD,1056,__LINEID__,Disk_6, Online, 1250.0, 0.0, , 
USSJCIRDB001PRD,1056,__LINEID__,Disk_7, Online, 1300.0, 0.0, , 
USSJCIRDB001PRD,1056,__LINEID__,Disk_8, Online, 300.0, 0.0, , 
USSJCIRDB001PRD,1056,__LINEID__,Disk_9, Online, 300.0, 0.0, , 
USSJCIRDB001PRD,1056,__LINEID__,Disk_10, Online, 100.0, 0.0, , 
USSJCIRDB001PRD,1056,__LINEID__,Disk_11, Online, 2500.0, 0.0, , 
USSJCIRDB001PRD,1056,__LINEID__,Disk_12, Online, 2500.0, 0.0, *, 
USSJCIRDB002PRD,1057,__LINEID__,Disk_0, Online, 136.0, 0.0, , 
USSJCIRDB002PRD,1057,__LINEID__,Disk_1, Online, 186.0, 0.0, , 
USSJCIRDB002PRD,1057,__LINEID__,Disk_2, Online, 2000.0, 0.0, , 
USSJCIRDB002PRD,1057,__LINEID__,Disk_3, Online, 300.0, 0.0, , 
USSJCIRDB002PRD,1057,__LINEID__,Disk_4, Online, 300.0, 0.0, , 
USSJCIRDB002PRD,1057,__LINEID__,Disk_5, Online, 2000.0, 0.0, , 
USSJCIRDB002PRD,1057,__LINEID__,Disk_6, Online, 100.0, 0.0, , 
USSJCIRDB002PRD,1057,__LINEID__,Disk_7, Online, 1000.0, 0.0, , 
USSJCIRDB002PRD,1057,__LINEID__,Disk_8, Online, 1000.0, 0.0, , 
USSJCIRDB002PRD,1057,__LINEID__,Disk_9, Online, 1000.0, 0.0, , 
USSJCIRDB003PRD,1058,__LINEID__,Disk_0, Online, 136.0, 0.0, , 
USSJCIRDB003PRD,1058,__LINEID__,Disk_1, Online, 800.0, 0.0, , 
USSJCIRDB003PRD,1058,__LINEID__,Disk_2, Online, 100.0, 0.0, , 
USSJCIRDB003PRD,1058,__LINEID__,Disk_3, Online, 800.0, 0.0, , 
USSJCIRDB003PRD,1058,__LINEID__,Disk_4, Online, 300.0, 0.0, , 
USSJCIRDB003PRD,1058,__LINEID__,Disk_5, Online, 300.0, 0.0, , 
USSJCIRDB003PRD,1058,__LINEID__,Disk_6, Online, 500.0, 0.0, , 
USSJCIRDB004PRD,1059,__LINEID__,Disk_0, Online, 136.0, 0.0, , 
USSJCIRDB004PRD,1059,__LINEID__,Disk_1, Online, 800.0, 0.0, , 
USSJCIRDB004PRD,1059,__LINEID__,Disk_2, Online, 100.0, 0.0, , 
USSJCIRDB004PRD,1059,__LINEID__,Disk_3, Online, 800.0, 0.0, , 
USSJCIRDB004PRD,1059,__LINEID__,Disk_4, Online, 300.0, 0.0, , 
USSJCIRDB004PRD,1059,__LINEID__,Disk_5, Online, 300.0, 0.0, , 
USSJCIRDB004PRD,1059,__LINEID__,Disk_6, Online, 500.0, 0.0, , 
USSJCIRDB005PRD,1060,__LINEID__,Disk_0, Online, 136.0, 0.0, , 
USSJCIRDB005PRD,1060,__LINEID__,Disk_1, Online, 800.0, 0.0, , 
USSJCIRDB005PRD,1060,__LINEID__,Disk_2, Online, 300.0, 0.0, , 
USSJCIRDB005PRD,1060,__LINEID__,Disk_3, Online, 800.0, 0.0, , 
USSJCIRDB005PRD,1060,__LINEID__,Disk_4, Online, 300.0, 0.0, , 
USSJCIRDB005PRD,1060,__LINEID__,Disk_5, Online, 800.0, 0.0, , 
USSJCIRDB005PRD,1060,__LINEID__,Disk_6, Online, 300.0, 0.0, , 
USSJCIRDB007PRD,1062,__LINEID__,Disk_0, Online, 136.0, 0.0, , 
USSJCIRDB007PRD,1062,__LINEID__,Disk_1, Online, 279.0, 0.0, , 
USSJCIRDB007PRD,1062,__LINEID__,Disk_2, Online, 250.0, 0.0, , 
USSJCIRDB007PRD,1062,__LINEID__,Disk_3, Online, 640.0, 0.0, , 
USSJCIRDB007PRD,1062,__LINEID__,Disk_4, Online, 100.0, 0.0, , 
USSJCIRDB007PRD,1062,__LINEID__,Disk_5, Online, 250.0, 0.0, , 
USSJCIRDB007PRD,1062,__LINEID__,Disk_6, Online, 100.0, 0.0, , 
USSJCIRDB007PRD,1062,__LINEID__,Disk_7, Online, 100.0, 0.0, , 
USSJCIRPF001PRD,1072,__LINEID__,Disk_0, Online, 279.0, 0.0, , 
USSJCIRPF002PRD,1073,__LINEID__,Disk_0, Online, 279.0, 0.0, , 
USSJCIRPF003PRD,1074,__LINEID__,Disk_0, Online, 136.0, 0.0, , 
USSJCIRPF003PRD,1074,__LINEID__,Disk_1, Online, 1676.0, 0.0, , 
USSJCIRWB001PRD,1075,__LINEID__,Disk_0, Online, 279.0, 0.0, , 
USSJCIRWB002PRD,1076,__LINEID__,Disk_0, Online, 279.0, 0.0, , 
USSJCIRWB003PRD,1077,__LINEID__,Disk_0, Online, 279.0, 0.0, , 
USSJCIRWB004PRD,1078,__LINEID__,Disk_0, Online, 279.0, 0.0, , 
USSJCIRWB004PRD,1078,__LINEID__,Disk_1, Online, 558.0, 0.0, , 
USSJCIRWB005PRD,1079,__LINEID__,Disk_0, Online, 279.0, 0.0, , 
USSJCIRWB005PRD,1079,__LINEID__,Disk_1, Online, 558.0, 0.0, , 
USSJCIRWB006PRD,1080,__LINEID__,Disk_0, Online, 136.0, 0.0, , 
USSJCIRWB006PRD,1080,__LINEID__,Disk_1, Online, 1676.0, 0.0, , 
USSJCIRWB006PRD,1080,__LINEID__,Disk_2, Online, 0.0, 0.0, , 
USSJCIRWB007PRD,1081,__LINEID__,Disk_0, Online, 136.0, 0.0, , 
USSJCIRWB007PRD,1081,__LINEID__,Disk_1, Online, 1676.0, 0.0, ,
 
Upvote 0
It looks like some of the words should remain joined e.g "service control manager". Should it be a fixed width delimit instead?
 
Upvote 0
That might work for this particular file but I'd be concerned about weird results on future versions. The most important thing is for the first few columns to delimit properly. So space delimiter seems like the best way to do that for this particular file.
But now that I think about it, since most of the files really are comma delimited its probably best to start with comma separation. After that I'll try to figure out how to deal with these exceptions.
 
Upvote 0
......
But now that I think about it, since most of the files really are comma delimited its probably best to start with comma separation. After that I'll try to figure out how to deal with these exceptions.


Hi
. I am a beginner, so please bear with me if I am missing the point. I prefer to use a simple VBA code to import a text file into Excel. If I apply such a VBA program to your CSV data then the first few rows imported into an Excel File look like this.


Book1
ABCDEFGHIJ
1hostnamehostuidlineidDisk_###StatusSize_GBFree_GBDynGpt
2USSJCIRCL001PRD1065__LINEID__Disk_0Online279.00.0
3USSJCIRCL002PRD1066__LINEID__Disk_0Online279.00.0
4USSJCIRCL003PRD1067__LINEID__Disk_0Online279.00.0
5USSJCIRCL004PRD1068__LINEID__Disk_0Online279.00.0
6USSJCIRDB001PRD1056__LINEID__Disk_0Online136.00.0
7USSJCIRDB001PRD1056__LINEID__Disk_1Online186.00.0
8USSJCIRDB001PRD1056__LINEID__Disk_2Online1000.00.0
9USSJCIRDB001PRD1056__LINEID__Disk_3Online1000.00.0
10USSJCIRDB001PRD1056__LINEID__Disk_4Online1000.00.0
11USSJCIRDB001PRD1056__LINEID__Disk_5Online1250.00.0
12USSJCIRDB001PRD1056__LINEID__Disk_6Online1250.00.0
13USSJCIRDB001PRD1056__LINEID__Disk_7Online1300.00.0
14USSJCIRDB001PRD1056__LINEID__Disk_8Online300.00.0
15USSJCIRDB001PRD1056__LINEID__Disk_9Online300.00.0
16USSJCIRDB001PRD1056__LINEID__Disk_10Online100.00.0
17USSJCIRDB001PRD1056__LINEID__Disk_11Online2500.00.0
18USSJCIRDB001PRD1056__LINEID__Disk_12Online2500.00.0*
19USSJCIRDB002PRD1057__LINEID__Disk_0Online136.00.0
MacroTextToExcel


. If that’s anywhere near wot you are looking for than check out the last Thread I answered on this Theme.

http://www.mrexcel.com/forum/excel-questions/795675-run-macro-txt-file.html?#post3917591

…. If I can help anymore than get back, but remember I don’t know much more than the very basics jet!!

Alan Elston.

P.s. I do not use the Split Function in my program as I only just read about that.- So as you see I am very much a beginner at the moment!!!
 
Upvote 0
Thank you very much for the reply DocAElstein, I can't quite get this to work, it is putting more than just the headers on the 1st row. Then it gives me runtime error 6 (overflow) on this line:
Code:
SpPs = InStr(TxtPs, DtaFleLn, Sp)
at SpPs position 251. I ran it as is with no revision to the code. Seems like it is not picking up where the line breaks and so not moving to
Code:
Let Rw = Rw + 1
.

In any case I ended up going with a very simplified version that works well enough for my needs:
Code:
Sub TxtImporter()

SpeedOn 'calls up a macro that turns off auto calculation, disables events, screen updating , etc... so the macro will complete faster

Range("A1").Select

'define variables:
Dim f As String, flPath As String
Dim i As Long, j As Long
Dim ws As Worksheet
i = ThisWorkbook.Worksheets.Count
j = Application.Workbooks.Count


'identify a specific cell on a specific sheet as the place where the folder path will be:
flPath = Sheets("START").Cells(8, 1).Value & Application.PathSeparator 'row 8, column 1 of sheet "START"
f = Dir(flPath & "*.csv") 'looking for csv files only in specified path
'loop through all csv files in the specified folder path:
Do Until f = ""

'list of exclusions:
If Not f = "EVENT_LOG.csv" Then
If Not f = "APP_LOG.csv" Then

'open each csv in excel, append to the workbook that contains this macro
    Workbooks.OpenText flPath & f, _
        StartRow:=1, DataType:=xlDelimited, TextQualifier:=xlDoubleQuote, _
        ConsecutiveDelimiter:=False, Tab:=True, Semicolon:=False, Comma:=True, _
        Space:=False, Other:=False, TrailingMinusNumbers:=True
    Workbooks(j + 1).Worksheets(1).Copy After:=ThisWorkbook.Worksheets(i)
    'ThisWorkbook.Worksheets(i + 1).Name = Left(f, Len(f) - 4)
    Workbooks(j + 1).Close SaveChanges:=False
    End If
    End If
    i = i + 1
    f = Dir
    'need to redefine the variables i & j after the loop passes over the files in the exclusions list. Otherwise the macro loses track of what these variables are supposed to be in the loop:
i = ThisWorkbook.Worksheets.Count
j = Application.Workbooks.Count

Loop

'calls up macro that sorts, subtotals, and does vlookup match on 2 of the new tabs:
Sort_Subtotal_Match

'Application.DisplayAlerts = True
Sheets("START").Select

SpeedOff 'calls up macro that undoes everything that SpeedOn did

MsgBox "import complete"
End Sub

This works ok except when the .csv is not actually comma separated. Some of the files I have are actually space separated despite being saved with .csv extension. Not sure what to do about that yet but I can get by with the above for now. Seems like your code might have the potential to detect what the actual separator is and do an if/then that runs a different version of code depending on what separator it detects. But I'm far off from being able to do that. Despite having hacked through VBA excel stuff for a while now I struggle mightily with text file import stuff ;) Thanks a again for your help!
 
Upvote 0
Thank you very much for the reply DocAElstein, ……….
clip_image001.gif
Thanks a again for your help!

Your welcome. And many thanks for the feedback and code.

……… I can't quite get this to work, it is putting more than just the headers on the 1st row. Then it gives me runtime error 6 (overflow) on this line:
Code:
SpPs = InStr(TxtPs, DtaFleLn, Sp)
at SpPs position 251. I ran it as is with no revision to the code. Seems like it is not picking up where the line breaks and so not moving to
Code:
Let Rw = Rw + 1
………..


. With how many Rows and Columns are you working?. (I tend to limit size of rows and columns in codes I give initially in the forum because of compatibility problems when I do not know which versions of Excel are in use).
. Which Excel version are you using?
.

Alan
 
Upvote 0
It varies. In this case the file is 9 columns and 73 rows but I need the macro to be able to import it regardless of the row/column count.
 
Upvote 0

Forum statistics

Threads
1,214,376
Messages
6,119,172
Members
448,870
Latest member
max_pedreira

We've detected that you are using an adblocker.

We have a great community of people providing Excel help here, but the hosting costs are enormous. You can help keep this site running by allowing ads on MrExcel.com.
Allow Ads at MrExcel

Which adblocker are you using?

Disable AdBlock

Follow these easy steps to disable AdBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the icon in the browser’s toolbar.
2)Click on the "Pause on this site" option.
Go back

Disable AdBlock Plus

Follow these easy steps to disable AdBlock Plus

1)Click on the icon in the browser’s toolbar.
2)Click on the toggle to disable it for "mrexcel.com".
Go back

Disable uBlock Origin

Follow these easy steps to disable uBlock Origin

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back

Disable uBlock

Follow these easy steps to disable uBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back
Back
Top