Blank Lines in CSV

[caass]

Hey everyone!

I have a csv that I am trying to clean in excel using VBA. If I open the file in notepad, it contains thousands of blank lines after my data ends which shouldn't be there. I want to flag the data and possibly automate deleting these rows in excel (amongst other data cleaning issues).

When I open in excel, there is only 900 rows of actual data. The problem is when I go to write the VBA, any formula I use to count the total number of rows is giving me 900 so I cannot figure out how to detect these extra rows using code.

If I go in manually and select the last row and ctrl+shft+down arrow and then delete, it then deletes the rows and shows that change in notepad. Is this just a problem because it is a csv? Is this an impossible task to automate?

 

[alansidman]

You may want to look at using Power Query/Get and Transform Data found on the Data Tab of the Ribbon to clean data. May be easier and more effective than VBA.

 

[antman2988]

Hi caass. Where is the CSV coming from? Are you creating it from an Excel file? Is it a queried file from a database? There could be a number of factors causing the blank rows. Also, are you able to upload a nonsensitive version of your CSV?

 

[Alex Blakenburg]

I am a little unclear on whether you really have an issue.
As I understand it the csv has 900 data rows and thousands of additional blank rows.

1) If the blank rows look like this in notepad
,,,,,,,,
Then when you import them into excel there is no need to delete them they will already be gone.

2) if they have anything in between the commas eg spaces then you will be able to see the extra lines when you import it into excel
ie
, , , , , , ,

From what I gather yours is scenario 1, and you are looking for additional lines that just aren't there.

One way to check it is to use ctrl+shift+8 (ctrl+*) to select what in vba is know as the current region.
In scenario 1 the current region will stop where the data ends
In scenario 2 it will also include your thousands of blank line.

 

[caass]

Hi caass. Where is the CSV coming from? Are you creating it from an Excel file? Is it a queried file from a database? There could be a number of factors causing the blank rows. Also, are you able to upload a nonsensitive version of your CSV?

Unfortunately I'm not able to upload a version of the CSV. It's coming from a database but something that is not in my control so I was looking for an alternative solution that I could handle myself but it seems like that might not be possible.

 

[caass]

I am a little unclear on whether you really have an issue.
As I understand it the csv has 900 data rows and thousands of additional blank rows.

1) If the blank rows look like this in notepad
,,,,,,,,
Then when you import them into excel there is no need to delete them they will already be gone.

2) if they have anything in between the commas eg spaces then you will be able to see the extra lines when you import it into excel
ie
, , , , , , ,

From what I gather yours is scenario 1, and you are looking for additional lines that just aren't there.

One way to check it is to use ctrl+shift+8 (ctrl+*) to select what in vba is know as the current region.
In scenario 1 the current region will stop where the data ends
In scenario 2 it will also include your thousands of blank line.

It is scenario 1. The lines aren't there in excel but when the file goes further down the line to production it's causing issues because of those blank rows. I think it's just something I need to push back on to fix in the early stages. Thank you!

 

[antman2988]

If you want to use a macro, you could try this. I haven't tested this so there could be issues. You may need to adjust the number of rows based on how many need to be deleted.

Sub DeleteBlankRows()
    Dim lastRow As Long

    lastRow = Cells.Find("*", , xlFormulas, , xlRows, xlPrevious, , , False).row
    If lastRow Is Nothing Then Exit Sub

    Rows(lastRow & ":" & lastRow+10000).EntireRow.Delete
End Sub

 

[antman2988]

Sub DeleteBlankRows()
    Dim lastRow As Long

    lastRow = Cells.Find("*", , xlFormulas, , xlRows, xlPrevious, , , False).row
    If lastRow Is Nothing Then Exit Sub

    With ActiveSheet   
        .Rows(lastRow & ":" & lastRow+10000).EntireRow.Delete
    End With
End Sub

 

[antman2988]

Sub DeleteBlankRows()
    Dim lastRow As Long

    lastRow = Cells.Find("*", , xlFormulas, , xlRows, xlPrevious, , , False).row
    If lastRow Is Nothing Then Exit Sub

    With ActiveSheet  
        .Rows(lastRow & ":" & lastRow+10000).EntireRow.Delete
    End With
End Sub

It should be .Rows(lastRow + 1....

 

[Alex Blakenburg]

Unfortunately I'm not able to upload a version of the CSV. It's coming from a database but something that is not in my control so I was looking for an alternative solution that I could handle myself but it seems like that might not be possible.

You can handle it yourself by simple opening the file and resaving it as csv.
Since just opening it in excel is removing the blank rows, when you resave it they won't be there.

We could write a macro to do it but the longest part of the process is just navigating to the folder and selecting the file.
If you have some parameters that would allow the macro to pick the right file or do all files in a folder, then you may save some time.

These are not solutions though. You really don't want excel in the middle of what should be an automated job ie Source database ----> Production system.
It really does need to be fixed at the source.