While AWS burns....
-
-
@pydsigner Is there context to the meme?
-
@RaceProUK I assume it's this:
http://venturebeat.com/2017/02/28/aws-is-investigating-s3-issues-affecting-quora-slack-trello/
-
http://venturebeat.com/2017/02/28/aws-is-investigating-s3-issues-affecting-quora-slack-trello/
I think this is also affecting WTDWTF's email.
“We are investigating increased error rates for Amazon S3 requests in the US-EAST-1 Region,” AWS said at the top of its status page.
-
I like the list of random places that's using this, then suddenly
the U.S. Securities and Exchange Commission (SEC)
-
The issues appear to be affecting Buffer, Business Insider, Chef, Citrix, Codecademy, Coursera, Docker, Expedia, Expensify, Giphy, Heroku, Home Chef, iFixit, IFTTT, isitdownrightnow.com, Lonely Planet, Mailchimp, Medium, Microsoft’s HockeyApp, News Corp, Quora, Razer, Slack, Sprout Social, Travis CI, Trello, Twilio, Unbounce, the U.S. Securities and Exchange Commission (SEC), and Zendesk, among other services. Airbnb, Down Detector, Freshdesk, Pinterest, SendGrid, Snapchat’s Bitmoji, and Time Inc. are currently working slowly.
So, no big sites the-
Microsoft’s HockeyApp
So much for dogfooding…
-
All sorts of things we use are down. Yup, the cloud will fix all. (wait, my dog was fixed...)
-
-
@RaceProUK said in While AWS burns....:
@pydsigner Is there context to the meme?
S3, and by proxy basically all of AWS and the internet, is down.
https://twitter.com/search?q=aws&src=typd
-
I meme'd:
-
Two reminders in one week why we shouldn't put up with this single point of failure bullshit.
-
@RaceProUK said in While AWS burns....:
The issues appear to be affecting ... isitdownrightnow.com
Yes.com
edit:
-
What I struggle to digest is not the outage. Those happen. What's crazy is that AWS's status page appears to rely... on S3. What kind of hubris does that demand?
-
Damn, shit just got real. I can't open my keepass db....
-
@dangeRuss said in While AWS burns....:
Damn, shit just got real. I can't open my keepass db....
-
@loopback0 said in While AWS burns....:
@dangeRuss said in While AWS burns....:
Damn, shit just got real. I can't open my keepass db....
What? Where do you store yours?
-
@pydsigner said in While AWS burns....:
What kind of hubris does that demand?
Probably the opposite. They are obsessive about dogfooding, which makes them subject to the same problems as their customers are.
-
@dangeRuss said in While AWS burns....:
What? Where do you store yours?
Several places. Hard drive. USB drive on my key chain. Phone. Dropbox.
-
@dangeRuss More than one place.
-
@loopback0 said in While AWS burns....:
@dangeRuss More than one place.
Well I do have a backup on dropbox, but it's a bit old. I do sync them once in a while.
My point was this is actually affecting me now.
-
-
@boomzilla said in While AWS burns....:
@pydsigner said in While AWS burns....:
What kind of hubris does that demand?
Probably the opposite. They are obsessive about dogfooding, which makes them subject to the same problems as their customers are.
I'm fine with dogfooding. That would be running amazon.com on AWS — which they don't do.
Running your status page on the systems the status page reports about? That's just foolishness or hubris.
-
@pydsigner said in While AWS burns....:
What's crazy is that AWS's status page appears to rely... on S3.
If the S3 status page works, S3 is up. You don't even need to write any code!
-
@pydsigner A total outage still technically counts as "Increased Error Rates"...
-
@pydsigner said in While AWS burns....:
I'm fine with dogfooding. That would be running amazon.com on AWS — which they don't do.
Running your status page on the systems the status page reports about? That's just foolishness or hubris.Oh. I don't know enough about it all to know that S3 is something different than AWS. I figured it was just some part of it.
-
We've been getting emails about this--our LMS uses AWS for all the uploads and file handling. When all the course content and homework is online, an outage like this is no fun at all.
-
@boomzilla said in While AWS burns....:
@pydsigner said in While AWS burns....:
I'm fine with dogfooding. That would be running amazon.com on AWS — which they don't do.
Running your status page on the systems the status page reports about? That's just foolishness or hubris.Oh. I don't know enough about it all to know that S3 is something different than AWS. I figured it was just some part of it.
S3 is only part of it, but, it's the backbone. Everything else depends on it. Including, apparently, parts of the status page
-
@boomzilla said in While AWS burns....:
US-EAST-1 Region
Ah, that explains why I've not seen it. Our stuff is in other regions (for legal reasons, despite the increased cost). You've not got a truly robust service if you don't replicate it in multiple AWS regions, since losing a whole datacenter in one go is entirely possible (perhaps due to a hurricane), and they really need to be geographically distributed. It increases the cost and complexity so a lot of people don't bother, but that's unavoidable if you really need that level of resilience.
The Clown never meant that you could shirk due diligence, despite all the hype.
-
-
@RaceProUK said in While AWS burns....:
The issues appear to be affecting...isitdownrightnow.com,
That's fucking hilarious.
-
I wonder if the DiscoCDNs are down.
-
@Benjamin-Hall said in While AWS burns....:
We've been getting emails about this--our LMS uses AWS for all the uploads and file handling. When all the course content and homework is online, an outage like this is no fun at all.
That's why we host our LMS locally.
-
@pydsigner said in While AWS burns....:
I wonder if the DiscoCDNs are down.
If they were, their upcoming solution will be resilience in the form of a DiscoCDNDN.
-
@dkf said in While AWS burns....:
The Clown never meant that you could shirk due diligence, despite all the hype.
-
@pydsigner said in While AWS burns....:
@boomzilla said in While AWS burns....:
@pydsigner said in While AWS burns....:
What kind of hubris does that demand?
Probably the opposite. They are obsessive about dogfooding, which makes them subject to the same problems as their customers are.
I'm fine with dogfooding. That would be running amazon.com on AWS — which they don't do.
Running your status page on the systems the status page reports about? That's just foolishness or hubris.
Actually, they do run Amazon.com on AWS now. It just doesn't use all the services - mostly just basic EC2.
-
@dkf said in While AWS burns....:
The Clown never meant that you could shirk due diligence, despite all the hype.
I've always been a bit surprised that the term "head in the Clouds" has never really caught on...
-
@dkf It seems to me that having one copy of your stuff in AWS and another copy in Azure in a different geographical region would basically guarantee 99.9999% uptime.
There should be a "meta-cloud" service to coordinate that for you.
-
@anonymous234 SaaSaaS?
-
@Weng said in While AWS burns....:
It just doesn't use all the services - mostly just basic EC2.
I've always operated that way too; basic EC2 (and EBS and S3) is something I grok, whereas the higher-level APIs are more confusing. I guess I don't think about services in the same way as a lot of other people do.
-
@anonymous234 said in While AWS burns....:
@dkf It seems to me like having one copy of your stuff in AWS and another copy in Azure in a different geographical region would basically guarantee 99.9999% uptime.
There should be a "meta-cloud" service to coordinate that for you.
There actually is. Or was. I read about it years ago.
-
@anonymous234 said in While AWS burns....:
@dkf It seems to me like having one copy of your stuff in AWS and another copy in Azure in a different geographical region would basically guarantee 99.9999% uptime.
There should be a "meta-cloud" service to coordinate that for you.
It's all fun and games until S3 goes down and your coordinating service 503s.
-
@pydsigner said in While AWS burns....:
It's all fun and games until S3 goes down and your coordinating service 503s.
That's what some of the more… unusual parts of DNS are for.
And your DNS provider probably runs in AWS.
-
-
@dkf If you're not building a highly complex network of 25 specialized cloud APIs calling each other in a loop, you're doing it wrong™.
(Mainly because this means you can migrate somewhere else easily, and Amazon does not want that)
-
@sloosecannon said in While AWS burns....:
@RaceProUK said in While AWS burns....:
isitdownrightnow.com
Uh.
Lol.
-
Must be hosted on S3
-
Hedgie FTW
-
@pydsigner said in While AWS burns....:
@anonymous234 said in While AWS burns....:
@dkf It seems to me like having one copy of your stuff in AWS and another copy in Azure in a different geographical region would basically guarantee 99.9999% uptime.
There should be a "meta-cloud" service to coordinate that for you.
It's all fun and games until S3 goes down and your coordinating service 503s.
Obviously the service is hosted on Google Cloud.
-
Looks like it was due to a typo. Oops.
-
@dcon said in While AWS burns....:
Looks like it was due to a typo.