FTP File



  • Our client has to download a file from a FTP Server from another company every 2 weeks.  The client wanted some special processing done on the file, so until I get time to automate the process, I've been manually downloading the file and processing it.  3 weeks ago, I was too busy or whatever and missed the file.  I figure no problem, it will be on the FTP Server still, since the name of the file always has a timestamp in it.  I look on the FTP and I see the next file but not the file I want.  I figure not a problem, I'll email the company and have them repost the file I need.  They repost the file, but it is the wrong one (they repost the next file which is still on the FTP).  So, I email them again and explain this.   Today, I get the following email.  

     <font color="#800080"><font size="2"><font face="Verdana">Hi {me},
     
    I've looked into requesting this "File Recreate", however based on the information I've received, there is a charge for this (possibly $1800).  Can you verify that you indeed need this recreate completed, and if so, would {our client} be funding this?  Are they aware?
     
    Thanks very much,</font></font></font>

     
    And, from an earlier email:
     

    <font color="navy" face="Arial" size="2">The previous cycle is no longer available as it was overlaid by the current cycle.<o:p></o:p></font>

     
     
    WTF,don't they have backups? And why are they deleting files, when each file has a unique name and they are only on the order of 200 KB and it is only a file every 2 weeks?   And why would it cost $1800?  This company had revenues of 43 billion dollar last year.  Can't they afford hard drives larger than 200 MB taht can hold all the files we'd ever need?  Very frustrating, because even though this company's policies are ridiculous it was technically my fault I missed the file...
     



  • @Salami said:

    Very frustrating, because even though this company's policies are ridiculous it was technically my fault I missed the file...

    True, but accidents happen.  This could have happened with an automated program as well.  This is why backups are so important.



  •  I know that we (well, one department) host regular report-type files for other customers. Thinks are a little different however - the customer logs into a secure web page (basically a bog standard username/password form over https) and then gets shown the reports for that month. They can however go to the parent and change to a previous month if they prefer. We keep the last year of reports or 1GB, whichever comes first, but with no data retention policy - if the hardisk fails holding the old data - tough, as the old data is considered obsolete and not worth backing up. Having a quota of 200KB/1 filedoes seem a little WTFy however, specially since the filenames are unique. 

    (We use a web form rather than FTP as FTP is considered insecure. I offered FTPS, but apparently thats "too complicated for the average customer". Maybe they are getting it confused with SFTP, but whatever, I don't care. They pay me to do the job, I do it. I've learned that trying to argue just gives me a headache.)



  • @Mole said:

    (We use a web form rather than FTP as FTP is considered insecure. I offered FTPS, but apparently thats "too complicated for the average customer". Maybe they are getting it confused with SFTP, but whatever, I don't care. They pay me to do the job, I do it. I've learned that trying to argue just gives me a headache.)
     

    SFTP and FTPS are no more difficult than FTP from a client-side perspective.  In FileZilla it's a matter of choosing "SFTP" (or "FTPS") instead of just "FTP" in the connection type.

    If the client is capable of adding an FTP site entry to FileZilla, they're capable of telling it to use SFTP instead.  If they're not, then the person setting it up for them is capable of doing it.



  • However, it'll probably take a week or even longer to fill in the required forms and get IT to install FileZilla on your PC in some companies (We have to predict how long it will take to install, projectected future support costs, risk analysis (whether the software could cause incompatabilities with other software,etc) and for software which accesses the internet an entire form on what kind of access it requires, how often, whether or not it needs to accept incoming connections (which is another frorm) etc). 



  • Unless they're using Windows Explorer to connect to FTP sites, chances are they're already using a client capable of SFTP, so I don't see it as much of an issue.  On the other hand, assuming they're using a standalone FTP client made in the last six years is probably an inexcusable leap of faith...

    Personally, if I were running a company providing data to my clients, I would simply not provide FTP access.  They can deal with paperwork on their end if they want to, I don't care - the instructions I give them are the same except for one additional letter.  If they refuse to use an encrypted protocol for transferring sensitive files, then I don't want their business in the first place.



  • It's worse than that, the clients I have dealt with (not sure about the rest) use Internet Explorer to browse FTP sites.

    Hence why we provide support over https as above. 



  • @Salami said:

    And why would it cost $1800?  This company had revenues of 43 billion dollar last year.

    And how do you think they got that 43 billion dollar revenue?  By charging reasonable prices for things?  Or by moving old files from their FTP Server's accessible directory to an inaccessible 'old files' directory (or maybe just marking the file invisible) and then charging ransom whenever a customer needs an old file?

    In any event, I'd advise you to at least automate the fetching of the file post haste.  (Note that I don't mean you should skip the quality control steps in your process, or you may find that you're overwriting the old data with new data, in addition to storing the new data.)



  • @Heron said:

    Unless they're using Windows Explorer to connect to FTP sites, chances are they're already using a client capable of SFTP, so I don't see it as much of an issue.  On the other hand, assuming they're using a standalone FTP client made in the last six years is probably an inexcusable leap of faith...

    Personally, if I were running a company providing data to my clients, I would simply not provide FTP access.  They can deal with paperwork on their end if they want to, I don't care - the instructions I give them are the same except for one additional letter.  If they refuse to use an encrypted protocol for transferring sensitive files, then I don't want their business in the first place.

    Your attitude is pretty unreasonable.  Money is money and you can't expect your customers to do what you want them to just because it's better for you.  If they want their data unencrypted and won't listen to your warnings, let them have it.  It's their data, after all.



  • Unreasonable?  I think not.  Who do you think is going to end up paying for it if their data gets stolen because we were using an insecure protocol?   It won't be the client, and I'm sure you're aware that small businesses usually don't have a very large legal budget.  Yes, it's their data, but they're going to hold me responsible for whatever happens to their data while it's in my possession - even if they're the ones who insisted on using the server that gets hacked (or whatever).

    If I'm having any sort of success at running my own business (that is, if I'm growing the business rather than stagnating), then I will be turning away clients once in a while anyway because I have too much work to be able to handle another client just then.  I'd rather make it clear up front that I do things securely or not at all, so that the clients I turn away are the ones who won't do things securely, rather than the ones who will.



  • FTP is actually industry standard.  The files on the FTP Server are all PGP encrypted.  SFTP secures the transmission, but PGP secures the file itself, which I think is more secure.  I deal with 50+ organizations and they all use FTP + PGP, except for 2 or 3 that insist on SFTP + PGP.  The industry is the electronic transfer of health claims, and other related data. 



  • @Salami said:

    FTP is actually industry standard.  The files on the FTP Server are all PGP encrypted.  SFTP secures the transmission, but PGP secures the file itself, which I think is more secure.  I deal with 50+ organizations and they all use FTP + PGP, except for 2 or 3 that insist on SFTP + PGP.  The industry is the electronic transfer of health claims, and other related data.
    Maybe I'm just playing devil's advocate here, but I don't think PGPing your files is any more secure than SFTP.  In addition to securing the transfer, you're securing the user's user/pass.  With FTP+PGP, you're potentially letting everyone into your encrypted files, but they could get to other places on your FTP server if you don't lock them down, or to a shell if you're not using an FTP-only account.  Not to mention publishing your public key somewhere ... public.  That same FTP directory would suffice since it's quite public.   

    Bottom line here is that SFTP is one thing to do (install ssh), whereas FTP+PGP is multiple things to do (install ftp, install pgp, generate key pair, encrypt every file, publish your public key, lock down FTP access to only that directory and hope it's not hacked).  This is of course just from the server side.  From the client side, you have to have them install filezilla (unless IE8/filezilla does sftp now) and teach how to use it versus installing php on the client side and teaching how to use it.  Not much difference from the client side, though I think most users would have an easier time learning Filezilla/whatever than PGP.  Not everything here applies to your situation, but it's the more general case.

    EDIT: two things for sftp.  switch account to SCPONLY.



  • @belgariontheking said:

    @Salami said:

    FTP is actually industry standard.  The files on the FTP Server are all PGP encrypted.  SFTP secures the transmission, but PGP secures the file itself, which I think is more secure.  I deal with 50+ organizations and they all use FTP + PGP, except for 2 or 3 that insist on SFTP + PGP.  The industry is the electronic transfer of health claims, and other related data.
    Maybe I'm just playing devil's advocate here, but I don't think PGPing your files is any more secure than SFTP.  In addition to securing the transfer, you're securing the user's user/pass.  With FTP+PGP, you're potentially letting everyone into your encrypted files, but they could get to other places on your FTP server if you don't lock them down, or to a shell if you're not using an FTP-only account.  Not to mention publishing your public key somewhere ... public.  That same FTP directory would suffice since it's quite public.   

    Bottom line here is that SFTP is one thing to do (install ssh), whereas FTP+PGP is multiple things to do (install ftp, install pgp, generate key pair, encrypt every file, publish your public key, lock down FTP access to only that directory and hope it's not hacked).  This is of course just from the server side.  From the client side, you have to have them install filezilla (unless IE8/filezilla does sftp now) and teach how to use it versus installing php on the client side and teaching how to use it.  Not much difference from the client side, though I think most users would have an easier time learning Filezilla/whatever than PGP.  Not everything here applies to your situation, but it's the more general case.

    EDIT: two things for sftp.  switch account to SCPONLY.

    Having your public key public is no problem at all.  Having the FTP password unencrypted isn't great, but without the key for the PGP file it's no biggie.  SFTP + PGP is more secure, but PGP-only is more secure than SFTP-only since the file can't be swiped if the server is compromised, the file will be encrypted once downloaded (the client can still store a decrypted version, but it's less likely) and finally the PGP key is likely more secure than any password would be used for SFTP.  SFTP can be forced to be key-only auth, but PGP is definitely key-only.



  • You could accept to pay the $1800 charge for recreating the file, except you bill that greedy company $2500 handling fees for writing the check.



  • @pebcak said:

    You could accept to pay the $1800 charge for recreating the file, except you bill that greedy company $2500 handling fees for writing the check.

     

     

    Not a bad idea, but they agreed to waive the fee one time.


Log in to reply