Combining Over a Hundred Text Files into One Text File

Or you could alternately title this entry:
What He Sed 🙂

The kinds of things that I typically want to do aren’t what most people would call normal. How many people that you know would ever come upon a situation where they would have three different directories which had more than sixty individual text files in each directory. And their end objective was to merge all of these text files into three different full text collections?

Copy/Paste you say?; or not for me. I just know enough to know that I can do it another way. A way that ‘would’ be easier and save time if I knew it off the top of my head. But ‘no’ I don’t know it off the top of my head so I have to research it for five hours until I can get the right combination to get it done … Would I have gone through with this if I had known it was going to take five hours? … ‘no’ probably not.

But when you are in the midst of the problem, one hour into it, you think you are close enough to get it finished in another half hour. But then after ninety minutes you think “Well I already have an hour and a half into it and I’m so close so I might as well finish it.”. And after three hours you say “I’m going to figure this out dag blame it!” 🙂

Now the aforementioned merging of all of the text files in the same directory would have been easy enough. I was given a little help by a new facebook friend in the Philippines. And after a couple of tries we were able to conclude that cat * >> alltext.txt would do it for me.

However a few of the text files were crowded up a bit at the beginning and the end of them and I just need to add three or four blank lines to the bottom of each of the text files before merging them. Ok sound easy enough right? But really this is the part that took me most of that five hours … So after several hours (excluding a couple of food breaks and a few breaks to give attention to my children) I searched a few dozen pages and read dozens of paragraphs.

I abandoned echo for a little while (a solution that had worked on a single file), went to sed (which I didn’t know much about), went back to echo when sed looked too difficult, went back to sed after trying several things with echo that didn’t work. Then looked over awk for a little while but thought that it was a whole different animal and went back to sed.

Does that sound convoluted enough? 🙂

After several more tries I finally came up with the following line that I executed on my test directory and worked on three random files which I tested and have all turned out correctly.

sed -i ‘$ s/$/ \n\n\n\n\n\n/’ *

whew !

Advertisements

About this entry