I am looking to find an eloquent way to concatenate all the files in a specific directory, basically making them into one. I created a script that loops through and appends everything to the new file name by way of cat $pathname/$i >> $pathname/$newfile.txt. As each item gets appended, I remove it. The $newfile.txt is actually in a different directory.
Depending on the UNIX/LINUS OS, this works great or does not work correctly. I am actually building a rather large file that will eventually get FTP'd to a website. In some cases, the first line gets chopped off when the concatenation is occurring. To fix this a appended a blank line at the end of each item.
On one particular box, I see the following messages in the jobstack log, my routine runs early in the morning:
cat: cannot open $pathname/$i
rm: $pathname/$i: No such file or directory
The $pathname is actually the fullpath and $i is something as follows:
Each piece of information in this id has substantial relevance but the bottom line is that it is vehicle information. The ultimate filename is 06192003_c999999_100_1_vehicles.txt. Sometimes, the name gets chopped up as well where instead of vehicles, we may have veh.
If anyone has any suggestions or comments, please do not hesitate. The fact that it works well in one UNIX implemenation and not in others is frustrating.
Thank you in advance.