How to make a list of files (new attempt)

(Sorry for the repeat. Firefox seems to work better than IE and much better than Safari with this forum software. Let's hope it doesn't get garbled upon posting.)

Almost ten years ago I was running some commercial software for my company. At one point I had to look through the backup script shipped with the program, and amidst some - afaik reasonably sound - bourne shell script code, my poor eyes (the goggles, they do nothing!) were locked on to the following section. I haven't slept properly since.

Now, some clarification is necessary. First, I have anonymized this by changing one directory name - the rest was copied verbatim. Second, the /data/dumps directory contains dump files only. No subdirectories. Third, the WTF is not that the author hasn't used all the cool new features of ksh and bash. This is *Bourne Shell*.

Enjoy:

#
# Find all files to write to tape
#
rm -f /tmp/files1
# dump-directory
find /data/dumps -name "*.dmp*" -print >> /tmp/files1
# make sure files are only backed up once
sort /tmp/files1 | grep -v '^ *$' > /tmp/files2
uniq /tmp/files2 /tmp/files3
echo dummy >/dev/null 2>/tmp/files9  # creates empty file
while [ `wc -l /tmp/files3 | awk '{print $1}'` -gt 0 ]
do
  filename=`line < /tmp/files3`
        echo $filename >> /tmp/files9
        grep -v "^$filename"$ /tmp/files3 > /tmp/files4
        grep -v "^$filename/" /tmp/files4 > /tmp/files5
        mv /tmp/files5 /tmp/files3
done
cat <<EOI > /tmp/awkscript
{filelist = filelist " " \$0}
END {print filelist}
EOI
files=`awk -f /tmp/awkscript /tmp/files9`

For those who aren't used to reading shell script code, I should explain that what this block of code does is to build a list of all the files in the /data/dumps directory, which is then used later on to backup those files to tape. The script does include some extra (though unnecessary) code for removing duplicates and subdirectories and stuff. The full functionality (as far as I can figure out) can be implemented thus:

files=`find /data/dumps -name '*.dmp*' -print | sed 's@^\(/data/dumps/[^/]*\)/.*$@\1@' | sort -u`

but in this particular case they could have just used

files=`ls /data/dumps/*.dmp* | sort`

Belcat

@Khim said:

rm -f /tmp/files1

dump-directory

find /data/dumps -name ".dmp" -print >> /tmp/files1

make sure files are only backed up once

sort /tmp/files1 | grep -v '^ *$' > /tmp/files2
uniq /tmp/files2 /tmp/files3

Using hard coded filenames like that is a great way to have random things overwritten... All someone has to do is:

ln -s /tmp/files2 /etc/passwd

That should be lots of fun. Now are you going to sleep at all? :)

@Khim said:

#
# Find all files to write to tape
#
find /data/dumps -name ".dmp" -print >> /tmp/files1
sort /tmp/files1 | grep -v '^ *$' > /tmp/files2
(more code)

but in this particular case they could have just used
files=`ls /data/dumps/*.dmp* | sort`<FONT face="Times New Roman">

</FONT>

<FONT face=Tahoma>Could this have been a workaround for the problem when ls complains that "arg list too long"?</FONT>

wiggzie

@Khim said:

I haven't slept properly since.

the wtf must be that you lost 10 years sleep over this...

m0ffx

That's a good point. Although xargs exists for that.