Anybody else been noticing the fairly frequent 504's over the past few days?
-
-
Ye
-
You mean like just now?
-
Noticed you were talking about this on meta I just updated the base container and docker which has better db configurations and a bunch of other fixes
@PJH @apapadimoulis be sure to PM me if you notice anything odd perf wise. If budget allows you can upgrade to a $40 instance and it will give you much more breathing room (though not required)
-
Cheers. Seems to be cleared up now. I guess the period where it was 404ing was when you were updating the thing.
Now, who was talking on meta without inviting me? There will be bitching and moaning in any case, so 'fess up!
-
[en.topic_count_all] Click to show.
-
Was offline for the last hour or so. Already started contemplating the sad state of my life and how much time I waste on meaningless online activities, when it came back. Crises averted.
-
Where is that, can you refresh browser, does it still happen?
-
-
Originally posted 1h45m ago:
Refreshing fixed it for me, but it isn't the first time I've seen it today.
-
No no, dude, you need the one with cute little cat who tripped and tore out the server's power cables.
-
So let me get this straight: You performed a non-critical software update that fucked up all your clients and put them offline for over an hour, at 5 pm EST on a Friday?
If that is not front page material, I don't know what is.
-
Invite how? TDWTF was down!
-
Main page (what.thedailywtf.com).
It says words in English now.
-
confirmed its fine on mobile was old js, in theory it should request new js when you navigate around
-
it's 7:23AM on sat in australia.
Sam, this has happened a day or two ago as well, are you aware of any maintenance at that point?
-
You saw me on meta! Don't lie! You even liked my post! But that like was a lie, I see it now!
-
Nope, but I suspect after running for 3 months somehow it got a bit tight on ram during an upgrade (during upgrade via web we need a bunch of extra memory to compile js and spin new processes)
-
I wasn't even aware of the discussion, TBH. So I wouldn't have been the one to invite you.
Have an extra like to show that other like was not a lie.
-
Why are you compiling JS on the client? It seems like the sort of thing that can be done once and deployed by Git or rsync.
Won't somebody think of the cycles?
-
I just had it happen again on mobile, after closing the tab when the downtime started and reopening it just recently.
-
Won't somebody think of the cycles?
On our roadmap to handle this better but its really complicated cause we need to amend the asset pipeline a lot, source maps also are also very high on our priority list in this department.
-
[en.topic_count_all] Click to show.
on main page.
-
[en.topic_count_all] Click to show.
TBH, that happens so much I don't even pay attention to it.
Pretty much any time something 503s or 404s unexpectedly, or you click a button too many times too fast, or you blink, Ember shits itself.
-
Don't blink. Don't even blink. Blink and it's dead.
-
+� reference acquired.
Well, using &#; doesn't work very well on low post numbers✐
-
Dicsourse broke again. And I thought this one had been fixed before:
In the topics list:
At the top of this topic:
I just ... I don't .. :sigh:
-
Yeah, in sure it isn't actually fixed like @Sam claims, as I keep getting it, even now.
-
Yeah, in sure it isn't actually fixed like @Sam claims, as I keep getting it, even now.
Just pop open the console and wait for JS errors when you click something as something else is auto-loading, or a race condition loses(wins?). It's a big bag of fail every time that happens.
-
I had it happen immediately after a full refresh, no navigation otherwise, on the first batch of new posts that came in.
It's off and on since I first saw it, sometimes happens, sometimes doesn't.
-
on the first batch of new posts that came in.
First batch of posts race conditioned each other ;)
-
Certain users appear to be flooding the site, @PJH have a look at
/logs
see env for the user there.
-
no, I performed a 5 minute emergency update, the one hour or whatever outage your are talking about is the reason I took some time on a saturday morning to update the site.
-
It's a big bag of fail
qftAlso...
On our roadmap to handle this better but its really complicated cause we need to amend the asset pipeline a lot,
So the solution isIf budget allows you can [spend more money on hosting this free software] and it will give you much more breathing room
And you still reckon you're close to v1? If we're using Burns on meta, the staff should be using Herschel Krustofski.Also also
its really complicated cause we need to amend the asset pipeline a lot,
Does that not mean you're Doing it Wrong™?
-
Certain users appear to be flooding the site, @PJH have a look at /logs see env for the user there.
... sorry @Sam, what is it I'm looking for?:
Yes - I know there's a user there, but which errors?
-
Certain users appear to be flooding the site, @PJH have a look at /logs see env for the user there.
I would just like to point out that if my name shows in those logs, there is an explanation and it is Discourse's fault and point you to another bug. While we were getting 504 errors yesterday, I refreshed and Discourse put itself in a refresh loop...and I left it. As soon as it happened, I (maybe wrongly) assumed that it was related to the stupid fucking infinite scroll, locked my computer and went to pick up my boy from daycare and just let it run. If you want to DOS your own software using my browser, who am I to judge?
-
A refresh loop sounds like a horrible bug, I will review the code. Regardless I am adding a haproxy rate limiter tomorrow into our base image which will protect this instance from a whole class of issues, looking at you @PaulaBean
-
I that errors thing look like a bug I will review it, but it look like someone has like post on a very aggressive bot. Regardless a rate limit here is really needed and I will add it.
-
Fellas, say goodbye to your like scripts.
-
A refresh loop sounds like a horrible bug, I will review the code.
I take it from that, that my username showed up on the logs?
-
Over 24 hours later, just had it happen to me. Main page; latest view. Also showed gray circles for topics I'd never even entered, yet. All OK on refresh.
-
This post is deleted!
-
but it look like someone has like post on a very aggressive bot. Regardless a rate limit here is really needed and I will add it.
they're only botting the race condition that allows duplicate likes per post.
-
@Intercourse said:
I would just like to point out that if my name shows in those logs, there is an explanation and it is Discourse's fault and point you to another bug. While we were getting 504 errors yesterday, I refreshed and Discourse put itself in a refresh loop...and I left it. As soon as it happened, I (maybe wrongly) assumed that it was related to the stupid fucking infinite scroll, locked my computer and went to pick up my boy from daycare and just let it run. If you want to DOS your own software using my browser, who am I to judge?
+1
-
AFK is a barrier to 504s.
-
They are happening again, very frequently for me. It is to the point that if I go to do something on here I am never certain if the request is going to go through or not. Now I copy each post off to the clipboard before submitting because it lost my post earlier when I attempted to submit it.
I Discurse Discourse.
-
I've seen stuff a couple of times today. I assumed the logs are filling up the disk again or something.
-
Maybe they are logging too much shit then?
-
I assumed the logs are filling up the disk again or something.
I know the bots got turned down due to that, but I also vaguely remember someone talking about using a 8k character user string to muck with things in one of the ideas threads so this is very possible.
-
Of use here might be the following:
Create a logger which ages the logfile once it reaches a certain size. Leave 10 “old” log files where each file is about 1,024,000 bytes.
logger = Logger.new('foo.log', 10, 1024000)