"Everything is live. Why would you need a test environment?"
-
This was the question I was just asked by a vendor.
See, about 6 months ago we had implemented an integration with a vendor. During this time, they gave us access to a sandbox API so we could get things set up, and organized it in the typical way where you'd have your dev env and integration tests configured to connect to the sandbox and prod env configured to connect to their prod API.
The implementation also included a webhook which they'd connect to, since some of our requests would be processed within 48 hours. Twice during development, I found a bug on their side where data we needed to access from their response was null, and such lack of data meant the system had to just throw its hands up and log an error, resulting in our users asking for it to see a sad "Server Error. Plz try again thx." response. They had fixed it promptly both times, and we were able to put it up on production.
Then, after we hit production, the same bug would crop up on their side, at least once a month, and only on their production API, so our integration tests to the sandbox passed. It got to the point I contemplated adding code to automatically email myself and our vendor contact when it hit the error. I was like do they do regression tests or anything? How is it the same bug keeps coming back every time they make a change to their code? Mind you, this isn't even an edge case bug. The data this call lacks literally the majority reason one would use their API.
Then, yesterday, we found that our integration tests started failing. Their sandbox API was no longer responding. I sent them an email asking if it was temporarily unavailable or they moved it to a different URL or something, and this was their response:
"We felt it was unnecessary to have to maintain two versions of our API, and so we have shuttered our test API environment. Besides, you have already launched your project and everything is live. Why would you need a test environment?"
Sadly, this vendor fills a niche, so there isn't even an alternative vendor to use. I'm almost inclined to have our company buy this vendor out just so we can fire their IT management and replace it with someone who knows what they're doing.
-
@The_Quiet_One at least you had a test environment to begin with...
We are currently building an integration with a vendor API for flight search and booking. When asked "how do we make a test booking?", their reply was "make a real booking and email us the number, we will cancel it".
-
@The_Quiet_One said in "Everything is live. Why would you need a test environment?":
I'm almost inclined to have our company buy this vendor out just so we can fire their IT management and replace it with someone who knows what they're doing.
-
@DCoder said in "Everything is live. Why would you need a test environment?":
@The_Quiet_One at least you had a test environment to begin with...
We are currently building an integration with a vendor API for flight search and booking. When asked "how do we make a test booking?", their reply was "make a real booking and email us the number, we will cancel it".
When I did my training for my tech support gig during my time at university, the "test environment" was located on the production system with some dummy accounts thrown in.
Then again, the whole thing was based on a Telnet interface, so...
-
@The_Quiet_One said in "Everything is live. Why would you need a test environment?":
buy this vendor out just so we can fire their IT management and replace it with someone who knows what they're doing.
Perhaps this is their investors' exit plan.
-
@Rhywden said in "Everything is live. Why would you need a test environment?":
Telnet interface
shudders
I deal with this shit still...
Filed under: At least we gave up on faxes
-
@DCoder said in "Everything is live. Why would you need a test environment?":
@The_Quiet_One at least you had a test environment to begin with...
We are currently building an integration with a vendor API for flight search and booking. When asked "how do we make a test booking?", their reply was "make a real booking and email us the number, we will cancel it".
We do similar thing on an e-commerce online ordering system. The testing is performed on the live system, with any user prefixed with "test". The order made by such accounts will not enter the backend for further processing.
-
@Onyx said in "Everything is live. Why would you need a test environment?":
I deal with this shit still...
As long as it is just dealing with running normal command-line programs that way, it's not too awful as there are reasonable tools for handling this. When you have to automate some nasty
GUITUI then you have the real pain. You get a little bit of it with running things like password-setting programs, but they're really minor by comparison with something that wants to pretend to be a lame-ass IDE…
-
@dkf It's actually an API.
It's not that horrible most of the time apart from having to deal with fragmented messages. There are also some instances where
\r\n\r\n
is actually not the end of the message as it should be by spec (because it's executing commands in a separate CLI application and passing the output along) so there's some message handling, but all in all I get by.Yes, I know it's not as bad as many had it before me, but I still want it dead and gone.
-
@The_Quiet_One said in "Everything is live. Why would you need a test environment?":
have our company buy this vendor out just so we can fire their IT management
-
@cheong said in "Everything is live. Why would you need a test environment?":
The testing is performed on the live system, with any user prefixed with "test". The order made by such accounts will not enter the backend for further processing.
New user: Why can't I order anything
Help: What's your user name?
New user: Testicular
-
@Onyx said in "Everything is live. Why would you need a test environment?":
@Rhywden said in "Everything is live. Why would you need a test environment?":
Telnet interface
shudders
I deal with this shit still...
So does pretty much every Linux sysadmin ever. (They call it "SSH" to sound modern and fancy, but it's still a Telnet connection.) And they actually like it! :o
-
@cheong said in "Everything is live. Why would you need a test environment?":
The testing is performed on the live system, with any user prefixed with "test". The order made by such accounts will not enter the backend for further processing.
How did you test that feature?
-
@Jaloopa said in "Everything is live. Why would you need a test environment?":
@cheong said in "Everything is live. Why would you need a test environment?":
The testing is performed on the live system, with any user prefixed with "test". The order made by such accounts will not enter the backend for further processing.
How did you test that feature?
Why? Of course we test a lot while we test other features. :P
-
@masonwheeler said in "Everything is live. Why would you need a test environment?":
And they actually like it!
Indeed we do.
It's taken MS decades to work out why that should be. They're still not quite getting it, but they're closer than they were.
-
@masonwheeler I really wanted to upvote you, but apparently SSH is actually more complex than that and supports binary streams of all sorts, so it's not really true.
-
@masonwheeler said in "Everything is live. Why would you need a test environment?":
They call it "SSH" to sound modern and fancy, but it's still a Telnet connection.
Except it's not.
-
@Gąska said in "Everything is live. Why would you need a test environment?":
@masonwheeler said in "Everything is live. Why would you need a test environment?":
it's still a Telnet connection.
Except it's not.
-
@flabdablet that "song" is burning my brain. I'm surprised I endured whole eight seconds of it.
Was it just random post or should I google lyrics because they're somehow relevant to the topic?
-
@Gąska for people using telnet: ssh is exactly like where you are right now, only much, much better.
-
@flabdablet How is ssh "much, much better" than telnet for the common user apart from encryption?
X11 tunnelling? Big whoop.
-
@anonymous234 Apart from encryption keeping plaintext passwords off the wire, X11 tunnelling, general-purpose port forwarding, ad-hoc VPN, key-based password-free auth, session multiplexing, proxying, command encapsulation and inbuilt file server and SOCKS5 proxy server... what have the Romans ever done for us?
NOTHING.
-
@anonymous234 said in "Everything is live. Why would you need a test environment?":
@masonwheeler I really wanted to upvote you, but apparently SSH is actually more complex than that and supports binary streams of all sorts, so it's not really true.
Yes, exactly. SCP/SFTP (not to be confused with FTPS) are a specific subsystem of SSH. SFTP is the only subsystem enabled by default in most SSH installs, but that doesn't preclude other ones from being written.
From the name SFTP, you'd think it's be an implementation of FTP running over SSH. If you assumed that, you'd be wrong; FTPS is FTP over SS
HL. SFTP was written as an entirely new protocol in the early to mid 2000s.
-
@powerlord said in "Everything is live. Why would you need a test environment?":
FTPS is FTP over SSH
FTP over SSL, generally. Even so, SSH also makes a perfectly serviceable wrapper for securing FTP if that's something you'd like to do with it.
-
This post is deleted!
-
@flabdablet Yeah whoops, I derped there.
-
@anonymous234 said in "Everything is live. Why would you need a test environment?":
@masonwheeler I really wanted to upvote you, but apparently SSH is actually more complex than that and supports binary streams of all sorts, so it's not really true.
So? Back in the day, I used to download binary files over a Telnet connection all the time.
-
@flabdablet said in "Everything is live. Why would you need a test environment?":
FTP over SSL, generally.
I've seen that in the wild. Once you've got a firewall in place, it's usually faster to use HTTPS.
-
@dkf About the only thing that FTP(S) has going for it over HTTP(S) is that FTP's separation of the data and control connections gives you real-time control over data behaviour; you don't need to wait for a crapload of oops! wrong file! to arrive before aborting it and transferring a different one.
This matters so infrequently that as advantages go it's pretty much negligible.
-
@flabdablet said in "Everything is live. Why would you need a test environment?":
real-time control over data behaviour
Except no clients seem to implement this anyway. Chunked HTTP can give the effect the other way too. Other purported benefits (restartable transfers, parallel transfers) are actually supported in HTTP too; just need a somewhat smarter than default client but then you also need that for those features in FTP. Feature-wise, it's a wash.
This matters so infrequently that as advantages go it's pretty much negligible.
QFT
-
@masonwheeler said in "Everything is live. Why would you need a test environment?":
So? Back in the day, I used to download binary files over a Telnet connection all the time.
That's no surprise given that FTP is based on Telnet.
... and then there are clients and servers that crash and burn if a telnet IAC arrives in the middle of something. And then there are other clients and servers that send some IACs every 1000 characters no matter what because the telnet standard says to do so...
-
@dcon said in "Everything is live. Why would you need a test environment?":
@cheong said in "Everything is live. Why would you need a test environment?":
The testing is performed on the live system, with any user prefixed with "test". The order made by such accounts will not enter the backend for further processing.
New user: Why can't I order anything
Help: What's your user name?
New user: TesticularThis reminds me a of an old story (of dubious authenticity) about how the earliest version of SABRE would shut down whenever someone tried to book a flight to the capital of Ecuador. Take a lucky guess what word was used as the exit command, and how long the input buffer was.
-
@Bulb said in "Everything is live. Why would you need a test environment?":
That's no surprise given that FTP is based on Telnet.
[citation needed]
RFC 959 has an extensive appendix outlining the history of the protocol, and no where does it mention being based on Telnet.
Or do you mean that both use ASCII, as does HTTP and many other protocols created at the time?
-
@dkf said in "Everything is live. Why would you need a test environment?":
@flabdablet said in "Everything is live. Why would you need a test environment?":>
This matters so infrequently that as advantages go it's pretty much negligible.
QFT
Btw, if you need to serve lots of big files (say in the situation of fileserver mirror), if the FTP server does offer command to create local checksum for the files, you don't need to rely solely on the size and timestamp to see if the file has been changed. (*nix filesystem does not have archive attribute that is automatically reseted whenever a file has been modified, you can have two files that has same filesize and write time but the content is different)
-
@quijibo said in "Everything is live. Why would you need a test environment?":
[citation needed]
RFC959, §2.2:
control connection:
The communication path between the USER-PI and SERVER-PI for the exchange of commands and replies. This connection follows the Telnet Protocol.RFC959, §2.3:
... The control connection follows the Telnet protocol. ...
@quijibo said in "Everything is live. Why would you need a test environment?":
RFC 959 has an extensive appendix outlining the history of the protocol, and no where does it mention being based on Telnet.
No, the history section is the wrong one. That one does not mention being based on Telnet, because it has absolutely nothing to do with history. It has a lot to do with present, though.
-
@The_Quiet_One said in "Everything is live. Why would you need a test environment?":
I'm almost inclined to have our company buy this vendor out just so we can fire their IT management and replace it with someone who knows what they're doing.
Rich man's solution to all problems.
-
@DCoder When I used to work for this shop we had another customer that wanted to add like a coupon thing on the end of their booking process. So I went to the dev site, pulled down a copy by FTP (no source control, because it was a waste of time) and tried making the changes on my machine. Nothing worked.
Asked my boss for help and then he said "oh yeah you have to make all the changes on the live environment". The DB and the webservices on live I couldn't access from my dev machine, so I just had to break their whole booking flow while I developed on the machine.
-
@lucas1 said in "Everything is live. Why would you need a test environment?":
When I used to work for this shop
Is that when you started drinking?
-
What niche? This could be free money.
-
@groo It made it worse.
-
@ScholRLEA said in "Everything is live. Why would you need a test environment?":
This reminds me a of an old story (of dubious authenticity) about how the earliest version of SABRE would shut down whenever someone tried to book a flight to the capital of Ecuador. Take a lucky guess what word was used as the exit command, and how long the input buffer was.
Yeah, sounds pretty dubious. Unless the mechanism for booking a flight was initiated by entering the name of the destination city as a top-level command, the shutdown command would not have been valid in context.
-
@cartman82 said in "Everything is live. Why would you need a test environment?":
@The_Quiet_One said in "Everything is live. Why would you need a test environment?":
I'm almost inclined to have our company buy this vendor out just so we can fire their IT management and replace it with someone who knows what they're doing.
Rich man's solution to all problems.
Not necessarily.
@masonwheeler said in "Everything is live. Why would you need a test environment?":
@ScholRLEA said in "Everything is live. Why would you need a test environment?":
This reminds me a of an old story (of dubious authenticity) about how the earliest version of SABRE would shut down whenever someone tried to book a flight to the capital of Ecuador. Take a lucky guess what word was used as the exit command, and how long the input buffer was.
Yeah, sounds pretty dubious. Unless the mechanism for booking a flight was initiated by entering the name of the destination city as a top-level command, the shutdown command would not have been valid in context.
Dubious, but sadly not impossible, given my first and second-hand experience.
The earliest version of SABRE was in the 1970s or 80s, so we're talking about the amber screen era, and when people would call someone to book a flight (or do it right at the airport like they were taking the train, ah, the good old days). The "real" story is probably that the CSR would often forget to get into the destination city prompt, and instead enter "quito" into the main prompt for the system, resulting in the quit command being run.
Less of a WTF, more of a PEBCAK, at least considering the contemporary finicky nature of computer UI's back in those days.
-
@The_Quiet_One said in "Everything is live. Why would you need a test environment?":
The earliest version of SABRE was in the 1970s or 80s, so we're talking about the amber screen era, and when people would call someone to book a flight (or do it right at the airport like they were taking the train, ah, the good old days).
Early 1960s, actually, according to Wikipedia, with roots back to the early '50s.
And then there was Eaasy SABRE, for use by consumers through CompuServe, GEnie and eventually AOL. I used it once or twice; my recollection is that it was far from eaasy (not even counting the blatant bias in favor of AA flights).
-
@The_Quiet_One Outside of the Likes thread, I don't recall ever seeing a post with 52 likes.
-
@jinpa said in "Everything is live. Why would you need a test environment?":
a post with
5253 likes
-
@dkf said in "Everything is live. Why would you need a test environment?":
@jinpa said in "Everything is live. Why would you need a test environment?":
a post with
5253545556 likes
-
"
Round and roundUp and up it goes; where it stops nobody knows"
-
@dcon said in "Everything is live. Why would you need a test environment?":
@cheong said in "Everything is live. Why would you need a test environment?":
The testing is performed on the live system, with any user prefixed with "test". The order made by such accounts will not enter the backend for further processing.
New user: Why can't I order anything
Help: What's your user name?
New user: TesticularBut which real world user would of the balls to adopt that user name?
-
@BernieTheBernie said in "Everything is live. Why would you need a test environment?":
@dcon said in "Everything is live. Why would you need a test environment?":
@cheong said in "Everything is live. Why would you need a test environment?":
The testing is performed on the live system, with any user prefixed with "test". The order made by such accounts will not enter the backend for further processing.
New user: Why can't I order anything
Help: What's your user name?
New user: TesticularBut which real world user would of the balls to adopt that user name?
We found 3,330 records for Testicles last name.
-
@acrow said in "Everything is live. Why would you need a test environment?":
@BernieTheBernie said in "Everything is live. Why would you need a test environment?":
@dcon said in "Everything is live. Why would you need a test environment?":
@cheong said in "Everything is live. Why would you need a test environment?":
The testing is performed on the live system, with any user prefixed with "test". The order made by such accounts will not enter the backend for further processing.
New user: Why can't I order anything
Help: What's your user name?
New user: TesticularBut which real world user would of the balls to adopt that user name?
We found 3,330 records for Testicles last name.
Testudo is also a pretty likely name.