How to make a list of files (new attempt)



  • (Sorry for the repeat. Firefox seems to work better than IE and much better than Safari with this forum software. Let's hope it doesn't get garbled upon posting.)

    Almost ten years ago I was running some commercial software for my company. At one point I had to look through the backup script shipped with the program, and amidst some - afaik reasonably sound - bourne shell script code, my poor eyes (the goggles, they do nothing!) were locked on to the following section. I haven't slept properly since.

    Now, some clarification is necessary. First, I have anonymized this by changing one directory name - the rest was copied verbatim. Second, the /data/dumps directory contains dump files only. No subdirectories. Third, the WTF is not that the author hasn't used all the cool new features of ksh and bash. This is *Bourne Shell*.

    Enjoy:

    #
    # Find all files to write to tape
    #
    rm -f /tmp/files1
    # dump-directory
    find /data/dumps -name "*.dmp*" -print >> /tmp/files1
    # make sure files are only backed up once
    sort /tmp/files1 | grep -v '^ *$' > /tmp/files2
    uniq /tmp/files2 /tmp/files3
    echo dummy >/dev/null 2>/tmp/files9  # creates empty file
    while [ `wc -l /tmp/files3 | awk '{print $1}'` -gt 0 ]
    do
      filename=`line < /tmp/files3`
            echo $filename >> /tmp/files9
            grep -v "^$filename"$ /tmp/files3 > /tmp/files4
            grep -v "^$filename/" /tmp/files4 > /tmp/files5
            mv /tmp/files5 /tmp/files3
    done
    cat <<EOI > /tmp/awkscript
    {filelist = filelist " " \$0}
    END {print filelist}
    EOI
    files=`awk -f /tmp/awkscript /tmp/files9`
    

    For those who aren't used to reading shell script code, I should explain that what this block of code does is to build a list of all the files in the /data/dumps directory, which is then used later on to backup those files to tape. The script does include some extra (though unnecessary) code for removing duplicates and subdirectories and stuff. The full functionality (as far as I can figure out) can be implemented thus:

    files=`find /data/dumps -name '*.dmp*' -print | sed 's@^\(/data/dumps/[^/]*\)/.*$@\1@' | sort -u`
    

    but in this particular case they could have just used

    files=`ls /data/dumps/*.dmp* | sort`
    


  • @Khim said:

    rm -f /tmp/files1

    dump-directory

    find /data/dumps -name ".dmp" -print >> /tmp/files1

    make sure files are only backed up once

    sort /tmp/files1 | grep -v '^ *$' > /tmp/files2
    uniq /tmp/files2 /tmp/files3



    Using hard coded filenames like that is a great way to have random things overwritten... All someone has to do is:

    ln -s /tmp/files2 /etc/passwd

    That should be lots of fun.  Now are you going to sleep at all? :)



  • @Khim said:

    #
    # Find all files to write to tape
    #
    find /data/dumps -name ".dmp" -print >> /tmp/files1
    sort /tmp/files1 | grep -v '^ *$' > /tmp/files2
    (more code)

    but in this particular case they could have just used

    files=`ls /data/dumps/*.dmp* | sort`<FONT face="Times New Roman">
    </FONT>
    <FONT face=Tahoma>Could this have been a workaround for the problem when ls complains that "arg list too long"?</FONT>


  • @Khim said:

     I haven't slept properly since.

     

    the wtf must be that you lost 10 years sleep over this...



  • That's a good point. Although xargs exists for that.


Log in to reply
 

Looks like your connection to What the Daily WTF? was lost, please wait while we try to reconnect.