huge file text editing

[Gates Is Antichrist]

This is a sort-of Excel question because I want to edit a .CSV file to add headers before importing it...

To text-edit a 90 meg file, would there be anything wrong with using Word? Wordpad? What would be the issues or drawbacks?

Also, if I may push my luck: do you know if those editors are "perfect" - i.e. do they convert each and every ascii characters, versus (silently - grrrr!) converting certain non-routine ones to something like a blank. (Maybe what I mean to be saying is, are they effectively hex editors, as opposed to text editors?)

I'm thinking about using good old Brief in DOS, but wondered if experts ever did serious data editing with the built in .DOC processors.

I'm Win2000 FWIW. Thanks.

 

[Damon Ostrander]

Hi 'Gates',

Because of the large size of the file, I wouldn't recommend using any editing application, and especially not one such as Word that is not set up to efficiently work with text files. I would recommend that you create just your header as a separate file, then concatenate the two files with a standard file copy. You can do this using the '+' to indicate concatenation. For example, if your header is heading1.txt and the csv file is mydata.csv, I believe that you can concatenate them onto mydata2.csv using

copy heading.txt+mydata.csv mydata2.csv

from the Windows Run prompt. I would expect this to be MUCH faster than using a word processing application.

 

[Gates Is Antichrist]

You're absolutely right. This was really a question that's been nagging me for a long time.

FWIW, I was able to do this via Word without pain on a 90 meg file, though I suspect that having 512 meg RAM was the reason it went so well. compare.com satisfied me that there were no unexpected side effects.

However, for this specific project, concatenation is king. Thanks for the shrewd advice. (All I would add is to ensure there's a carriage return/linefeed, hex 0D0A)
_________________
Because he is.
This message was edited by Gates Is Antichrist on 2002-09-26 16:30