The long way to India
-
My company outsources certain development services to a company in India. As I generally am the point-person for our Indian rep at my company, and am in charge of QC in the US, I do a lot of interacting with the files they produce.
The files/product/program/whatever they are making are a few gigs in size, and I download them by connecting to an FTP server in India.
The problem is that my company has 1 3Mb bonded T1 connection, and over 100 computers. If a single person is using the network to do anything bandwidth intensive (like uploading a file to a vendor, or streaming music/video), nobody else can get speeds above 15KBps. People uploading files to various FTP servers is a very common occurrence. What's worse is that when the network is under high load, the connection to the Indian FTP server drops down to 1.5KBps. So downloading/uploading project files can take 13 hours or more if you're lucky. Combined with the time difference, this meant that these projects would have had to have their deadlines pushed back by at least a month.
After the first couple 13 hour downloads, I set my phone up to ring when I received emails from India, and would get up at 2-3AM to begin downloading the files at home and talking to our Indian rep. My home connection is 20Mb Down/5Mb Up and I pay 10 times less for it than my company does for a 3Mb D/1.5Mb U connection. From my home connection, the 13 hour download is reduced to 10-30 minutes. With similar speed gains for uploads. My commute is over an hour each way, and it still would have been twice as fast for me to drive home from work, download the files, and drive back than it would be using the company network.
I eventually refined the process even further so that I don't have to get up at odd hours of the morning. I took a company laptop that nobody would miss, set it up at home, and it now acts like a remote upload/download server I can use for whatever I need while at work.
In order to get files downloaded from the Indian FTP server, modified, and then uploaded again, I now do the following:
- Remote connect to home computer
- Connect to Indian FTP
- Download files
- Transfer the files over the network from my home computer to my work computer
- Transfer files from my work computer to the company server
- QC, and make modifications over the course of several days
- Transfer updated files from the company server to my work computer
- Remote connect to home computer
- Transfer the files over the network from my work computer to my home computer
- Connect to Indian FTP
- Upload files to Indian FTP
- GOTO 1
That process has reduced the total upload/download time by about 90%.
I did get a raise partly for coming up with a solution and getting everything QCd within the tight deadline - and now have job security as nobody else understands how I am able to get my projects done so quickly. So, even though my solution isn't particularly elegant, it's a WTF that worked out for the best.
-
Why not just install rsync?
-
@jpa said:
Why not just install rsync?
The files we are sending back to them are not necesarily even the same ones we downloaded from them originally (with modifications), but are often completely different files. Likewise, each iteration of the files sent to us from the vendor are not necessarily set up in the same way as the originals, or may only be certain pieces of different file sets. We also maintain separate copies of each major release, and sometimes separate copies of individual changes.While rsync might work in some cases, there is a lot of manual process involved in working with these files. Also - my solution only relies on me understanding how to transfer the files and takes very little of my time. Setting up something like rsync and getting the vendor to use it would be way too much time and trouble for me to go through compared to the currently working solution.
-
Wait... so they're willing to spend more money to solve the problem, as they gave you a raise, right? So why don't they just spend the money on fixing their crappy internet connection and make everyone in the company more productive?
-
@dcardani said:
Wait... so they're willing to spend more money to solve the problem, as they gave you a raise, right? So why don't they just spend the money on fixing their crappy internet connection and make everyone in the company more productive?
At the time, a raise for me was way way cheaper than getting a better connection.I did manage to eventually convince them to upgrade (they're getting a 10Mb connection), but the price is astronomical (like $1k+ or so a month). The company has very limited ISP choices, and they have little option but to pay the exorbitant prices.
My home connection will still be twice as fast. And, despite the upgrade, it will still take forever to do these file transfers to/from India at work, I imagine. So my remote computer solution will still be used, and I'll still get my work done faster than other people would be able to because of it.
-
@KrakenLover said:
- Transfer the files over the network from my home computer to my work computer
Wait, umm, how is this step any quicker than directly downloading the file to the work computer? Does the VPN get priority over normal internet traffic?
-
@random_garbage said:
@KrakenLover said:
- Transfer the files over the network from my home computer to my work computer
Wait, umm, how is this step any quicker than directly downloading the file to the work computer? Does the VPN get priority over normal internet traffic?
Glad you asked, I was confused about this too... are you missing a step where you physically carry the laptop into work?
-
@blakeyrat said:
@random_garbage said:
This is why it took so long for me to come up with the solution - because it doesn't make sense to me.@KrakenLover said:
- Transfer the files over the network from my home computer to my work computer
Wait, umm, how is this step any quicker than directly downloading the file to the work computer? Does the VPN get priority over normal internet traffic?
Glad you asked, I was confused about this too... are you missing a step where you physically carry the laptop into work?
For some reason, under periods of high use, all network traffic except connections between internal and external systems using whatever protocol Apple Remote Desktop uses for file transfers are excrutiatingly slow, and FTP in particular (any FTP connection) at work is much slower than it should be no matter what. Even when there is very little load on the company network, FTP connections to India typically drop down to 5-20Kbps, while at home they are several hundred Kbps or 1-2 Mbps.
I realize that my solution for getting around slow file transfers doesn't make much sense, but it's the best one that I have been able to come up with which works 100% of the time (I tired alternatives, but this way is simply the fastest and easiest), and that's part of the WTF.
I honestly don't know exactly why my solution works, but it does. It's as if our network capacity inexplicably increases when I transfer files in this manner - since I don't have access to any of our internal network equipment, I can't say what the priorities are for the different kinds of traffic. If I had to guess, I'd say FTP is at the lowest priority.
Whatever the case, and whatever the reason, downloading files on my remote machine (not just FTP, but any file) and then transferring the file(s) to my work computer over a remote desktop connection is faster 90% of the time than just downloading the file directly to my work computer from the Internet.
-
@KrakenLover said:
For some reason, under periods of high use, all network traffic except connections between internal and external systems using whatever protocol Apple Remote Desktop uses for file transfers are excrutiatingly slow, and FTP in particular (any FTP connection) at work is much slower than it should be no matter what. Even when there is very little load on the company network, FTP connections to India typically drop down to 5-20Kbps, while at home they are several hundred Kbps or 1-2 Mbps.
I'm not familiar with the said protocol, but if it uses UDP, then it's not really surprising as there isn't automatic congestion control like with TCP. And if there isn't, it becomes even worse for the rest of the users.
-
@KrakenLover said:
I honestly don't know exactly why my solution works, but it does. It's as if our network capacity inexplicably increases when I transfer files in this manner - since I don't have access to any of our internal network equipment, I can't say what the priorities are for the different kinds of traffic. If I had to guess, I'd say FTP is at the lowest priority.
Whatever the case, and whatever the reason, downloading files on my remote machine (not just FTP, but any file) and then transferring the file(s) to my work computer over a remote desktop connection is faster 90% of the time than just downloading the file directly to my work computer from the Internet.
That sounds like another proxy from hell to me. Maybe you gotsome kind of firewall that automatically scans all files transferred over FTP for viruses? Does something like that exist at all or am I being stupid?
Alternatively, could connections priorized differently depending on whether you or the remote site initialted the connection? Have you tried if passive FTP makes any difference?
Also, your "workflow" sounds like the perfect opportunity for a shellscript.
-
@Buzer said:
I'm not familiar with the said protocol, but if it uses UDP, then it's not really surprising as there isn't automatic congestion control like with TCP. And if there isn't, it becomes even worse for the rest of the users.
I looked at lists of the protocols used by ARD before, but file transfer is a TCP protocol, not UDP (assuming I read the list correctly, as there was also a "trivial file transfer protocol" that was UDP, but there is no explanation of under which circumstances that one is used rather than their other TCP protocol).@PSWorx said:
If there were consistency in file names and whatnot, then a script might work. But, as it is, I have to read the email that the vendor sends me along with the file to determine what files are where, what they are called, and what changes were made. It really doesn't take me long to download/upload the stuff - may five minutes of my time, and the rest is just waiting for the transfer to complete.Also, your "workflow" sounds like the perfect opportunity for a shellscript.
Side note, this forum software: WYSIWTF Does anybody here like it?
-
@KrakenLover said:
Side note, this forum software: WYSIWTF Does anybody here like it?
We all love it, and now you can never be part of our gang!
-
@KrakenLover said:
If there were consistency in file names and whatnot, then a script might work. But, as it is, I have to read the email that the vendor sends me along with the file to determine what files are where, what they are called, and what changes were made. It really doesn't take me long to download/upload the stuff - may five minutes of my time, and the rest is just waiting for the transfer to complete.
Well, I didn't mean actual syncing. Just e.g. a "download" and an "upload" script that you call manually with a file path on the FTP server and that automate steps 1 to 5 and 7 to 11, respectively. But as you write that doesn't seem to be your problem, so nevermind.
@blakeyrat said:
@KrakenLover said:
Side note, this forum software: WYSIWTF Does anybody here like it?
We all love it, and now you can never be part of our gang!
It's special in its own way!
-
@PSWorx said:
Special is exactly the word I had in mind.@blakeyrat said:
@KrakenLover said:
Side note, this forum software: WYSIWTF Does anybody here like it?
We all love it, and now you can never be part of our gang!
It's special in its own way!
-
Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway. Or a dude with a laptop bag on his morning commute.
Incidentally, in Canada, it is now cheaper and faster to overnight ship brand new solid-state drives loaded with data than it is to download it at their overage rates. ... And then throw the SSDs away.
-
@KrakenLover said:
Side note, this forum software:
Apart from the occasional slipups when I type < instead of <, I have no issues with it.
WYSIWTF Does anybody here like it?
(Last instance was when I replaced a whole quoted section with <stuff> instead of <stuff> and all that happened was an empty quote which looked crap. I didn't look that closely at that bit on the preview. Corrected when I saw the post when it came through email though.)
-
@KrakenLover said:
I looked at lists of the protocols used by ARD before, but file transfer is a TCP protocol, not UDP (assuming I read the list correctly, as there was also a "trivial file transfer protocol" that was UDP, but there is no explanation of under which circumstances that one is used rather than their other TCP protocol).
Education time:
TFTP is used for network boot. Often the machine requesting the boot image does not even have any local storage (hard disk), so the TFTP protocol is embedded in a boot ROM. This means that the protocol must be kept simple ("trivial").
-
@KrakenLover said:
I did manage to eventually convince them to upgrade (they're getting a 10Mb connection), but the price is astronomical (like $1k+ or so a month). The company has very limited ISP choices, and they have little option but to pay the exorbitant prices.
I don't get this.
What astronomical upscale of service/reliability/whatever do you got for that kind of money?
My guess: nothing.
-
@KrakenLover said:
For some reason, under periods of high use, all network traffic except connections between internal and external systems using whatever protocol Apple Remote Desktop uses for file transfers are excrutiatingly slow,
Honnestly, this sounds like you've got highly compressable data, and that the "Apple Remote Desktop" protocol implements a rudimentry compression scheme. If you got a raise for the rube goldberg machine you described, then i want to work for your company. By those standards i'd own the place in a month.
-
@dhromed said:
@KrakenLover said:
I did manage to eventually convince them to upgrade (they're getting a 10Mb connection), but the price is astronomical (like $1k+ or so a month). The company has very limited ISP choices, and they have little option but to pay the exorbitant prices.
I don't get this.
What astronomical upscale of service/reliability/whatever do you got for that kind of money?
My guess: nothing.
My guess is SLA. Consumer connections generally have none. And decent-sized IP block/routing for their own (many ISPs tend to want extra for routing when the company has obtained their own IP block for RIR)
-
I'm fairly confident that on any sizable company the internet connection starts to suck.
-
@KrakenLover said:
I did manage to eventually convince them to upgrade (they're getting a 10Mb connection), but the price is astronomical (like $1k+ or so a month). The company has very limited ISP choices, and they have little option but to pay the exorbitant prices.
Faced with a similar situation, we instead got a DSL line ran in and moved all our users on to it. There was no point in paying thousands more to upgrade our T1 when the majority of the traffic was from users streaming radio and watching videos. Now I have a nice T1 that's always available for servers and equipment to do real internet work and the users can fight for internet traffic over the DSL line. Also, since the DSL is a 6mbit connection sometimes I can download those huge files faster on it during non-peak hours.
-
@Buzer said:
My guess is SLA. Consumer connections generally have none. And decent-sized IP block/routing for their own (many ISPs tend to want extra for routing when the company has obtained their own IP block for RIR)
Bingo.
-
@error_NoError said:
Faced with a similar situation, we instead got a DSL line ran in and moved all our users on to it. There was no point in paying thousands more to upgrade our T1 when the majority of the traffic was from users streaming radio and watching videos. Now I have a nice T1 that's always available for servers and equipment to do real internet work and the users can fight for internet traffic over the DSL line. Also, since the DSL is a 6mbit connection sometimes I can download those huge files faster on it during non-peak hours.
My company is in a business district - so there is no residential service. We basically only have a choice of two high priced ISPs.We're right next door to a Boeing installation; I don't imagine they have trouble paying the high prices for their connection. But it's a burden to us.
-
@KrakenLover said:
I honestly don't know exactly why my solution works, but it does. It's as if our network capacity inexplicably increases when I transfer files in this manner - since I don't have access to any of our internal network equipment, I can't say what the priorities are for the different kinds of traffic. If I had to guess, I'd say FTP is at the lowest priority.
Whatever the case, and whatever the reason, downloading files on my remote machine (not just FTP, but any file) and then transferring the file(s) to my work computer over a remote desktop connection is faster 90% of the time than just downloading the file directly to my work computer from the Internet.
I would guess it's probably because the company that provides Internet to your business is either the same as or shares a network with the company that provides Internet to your house. I've occasionally found that transferring files to someone near me who gets their Internet from the same company will go faster than our theoretical maximum transfer rate, simply because the packets never actually leave the ISP's network and thus get moved around far faster than they "should".
After all, there's no sense in the ISP rate-limiting you when you're only using internal bandwidth; it's just a single step up from being on the same LAN, and costs about as much.
-
@Tacroy said:
We have Covad at work, and I have Verizon at home - I don't know if they have anything to do with one another in that respect, but I wouldn't imagine they do.I would guess it's probably because the company that provides Internet to your business is either the same as or shares a network with the company that provides Internet to your house. I've occasionally found that transferring files to someone near me who gets their Internet from the same company will go faster than our theoretical maximum transfer rate, simply because the packets never actually leave the ISP's network and thus get moved around far faster than they "should".
After all, there's no sense in the ISP rate-limiting you when you're only using internal bandwidth; it's just a single step up from being on the same LAN, and costs about as much.
-
@Tacroy said:
After all, there's no sense in the ISP rate-limiting you when you're only using internal bandwidth; it's just a single step up from being on the same LAN, and costs about as much.
Dunno about rest of the world, but at least in Finland the limiting is generally done at first possible place. Most of time that is at the modem (the modem & DSLAM/CMTS negotiate the speed, modem does it's best to be as close as possible and DSLAM/CMTS (or some device right after it) enforce the limit. I have read that in the past people hacked their cable modems to use higher speed configs, but I doubt that is possible with modern CMTSs).
-
Join the club. Where I work the only option we have for internet is T1 service. 1.5Mbps down/up per line and each line costs 300 bucks a month. I pay 35 bucks a month at my apt for 25 down and 2 up. Whenever I download any ISOs or anything for work, I just download them at home and use a flash drive or external HD for transfer.
The upside is that I live like 5-10 minutes from work, including time taken to walk to my car, etc.
-
@Volcanon said:
The upside is that I live like 5-10 minutes from work, including time taken to walk to my car, etc.
I live 40 or so miles away from my primary workplace. I can drive home and back to download a file and it would still be quicker than downloading it at work.
We do so much uploading/downloading to/from different vendors it's extremely frustrating for everybody. Every time the network is slow,we have to figure out who is the person responsible (if any) or if there is a bigger network issue. Get's pretty tiring.
-
@boh said:
@KrakenLover said:
I looked at lists of the protocols used by ARD before, but file transfer is a TCP protocol, not UDP (assuming I read the list correctly, as there was also a "trivial file transfer protocol" that was UDP, but there is no explanation of under which circumstances that one is used rather than their other TCP protocol).
Education time:
TFTP is used for network boot. Often the machine requesting the boot image does not even have any local storage (hard disk), so the TFTP protocol is embedded in a boot ROM. This means that the protocol must be kept simple ("trivial").
Also TFTP is often used as a fail safe for routers and other equipment, sometimes you can unbrick a rounter by flashing it via TFTP.
-
Apple Remote Desktop uses TCP 5900 for screen observe/control and TCP/UDP port 3283 for just about everything else (including copying files.) 3283 used to be used for all ARD communication until Apple wanted to also have ARD 2.x and higher be able to control screens that were using third-party VNC.
