You merely adopted the 500. I was born in it, molded by it. I didn't see a fully rendered Discourse page until I was already a man


  • Discourse touched me in a no-no place

    @TwelveBaud said:

    :slowpoke.tif:

    Some of us have work to do and are reading around that.


  • :belt_onion:

    @tarunik said:

    (we're mostly an Oracle shop, though -- our management wants someone to hold accountable if a database hotfix decides to halt trains for a day instead of, you know, fixing things).

    so wait, are they trying to CAUSE databases to fail and halt trains?
    Oracle's idea of accountability is to tell you that you should upgrade to version eleventybillion gee.



  • [Quote]Like, you know the user's going to click the damned speech bubble, pre-load that fucker![/quote]Are you crazy? That would waste bandwidth! Which they need to send every post id in the thread. Priorities.


  • Banned

    I am looking at fixing a few queries first, there are a bunch of issues I isolated here I would like to address first. (which means we are going to have to wait till next beta)


  • FoxDev

    fair enough then!

    Thanks for your time and effort.



  • @darkmatter said:

    so wait, are they trying to CAUSE databases to fail and halt trains?Oracle's idea of accountability is to tell you that you should upgrade to version eleventybillion gee.

    We're big enough that I suspect Larry Ellison doesn't want us putting coal in his stocking for Christmas... ;)



  • Do it anyway, for those of us with the misfortune of having to deal with Oracle and unable to do anything about it.

    (Okay, it's been 5 years since I've had to touch Oracle, but still....shudders)




  • Discourse touched me in a no-no place

    @TwelveBaud said:

    .. expecting those features to work properly... well, with Discourse, they won't.

    Shocker.


  • Java Dev

    The real question is: Does your CO play golf with Oracle's CO?


  • Banned

    We just released a new beta, not promising anything but it does improve a few areas.

    @PJH I am running the upgrade now, I want to rebuild the container and enable a bit more diags for me.


  • Banned

    We are upgraded now



  • That explains all the "err_connection_refused" that happened just a few minutes ago.


  • I survived the hour long Uno hand

    Fun fact: I couldn't read the notice about the site going down because it went down.



  • Quoth the Duckhorse: 504



  • You merely adopted the 500. I was born in it, molded by it. I didn't see a fully rendered Discourse page until I was already a man

    Ok, I'm not into meta, but I LOL-ed at this. Whoever made the edit, kudos.


  • Discourse touched me in a no-no place

    Kudos to @jaloopa then



  • @RaceProUK said:

    Jeff will just blame the bots, or t/1000, or servercooties.com,
    You know, that blame ended up being correct, even if not complete...@sam said:
    I want to rebuild the container and enable a bit more diags for me.
    Please keep us posted on what you find out, and when we can/should move to a more managed provider. Thanks!


  • FoxDev

    @TwelveBaud said:

    You know, that blame ended up being correct, even if not complete

    It is true we effectively DoS'd the site when we started polling t/1000, and since turning off the Readify modules on my bots, we've had less issues (since they're no longer readifying t/1000)… but also that doesn't excuse the base issue, which is definitely in Discourse.


  • kills Dumbledore

    @loopback0 said:

    Kudos to @jaloopa then

    I think it's my favourite title edit I've done


  • Banned

    Minor outage ahead while I twiddle with

    and rebuild.

    Getting tons and tons of 500 errors atm.


  • FoxDev

    So… what exactly is that meant to do?


  • Banned

    redis was chucking an error causing a 500 on bgsave despite having a gig and a half of free ram.

    It needed this overcommit flag on the kernel, I will be amending ./launcher to add a pre-req test for this setting cause it a rather bad failure mode for no legit reason.


  • BINNED

    @RaceProUK said:

    So… what exactly is that meant to do?

    Realign the quantum forbinators?

    More realistically, allow Redis to overcommit? I'm guessing it would be letting it allocate more memory than configured? Why couldn't you just the bump up the value in config...

    @sam said:

    redis was chucking 500 on bgsave despite having a gig and a half of free ram.

    Ah, broken. Gotcha.


  • FoxDev

    @sam said:

    redis was chucking 500 on bgsave despite having a gig and a half of free ram.

    *looks up redis and bgsave*

    …oh… yeah… don't really want that broken…


  • Banned

    I just commented out this line:

    thing is ... rebaking all the posts you are quoted in may be interesting in theory each time you change avatars. it needs some severe rate limiting in practice, especially here. CPU was pegged cause there was a 4 way rebake going for all the posts a certain user here who changed avatars made, we will redesign so this can not happen.



  • DDOS by avatar, that's new...
    Also: weren't you on vacation? Stop wasting your free time here, please...



  • @apapadimoulis said:

    What the fuckity fuck? I hate modern software development.

    Last time I expressed any sentiment even vaguely close to that, I was accused of "fearing change".

    Don't worry about it, Alex. The only thing worse than getting older is not getting older.



  • @sam said:

    T/1000 is fucking us big time

    So, the only thread on this forum that would actually be a pain in the arse to navigate via paging turns out to be even more trouble given infiniscroll?

    AMAZEBALLS

    How many times was Jeff told that infiniscroll is a non-solution looking for a problem to solve? Not enough times. That's how many.



  • @TwelveBaud said:

    Jeff does not want to fix it because it was generated by artificial traffic in an artificial topic; IMO he's viewing it through the lens of "those damn TDWTFers, tryin' to break my stuff!"

    Sam does want to fix it because it could come up in an organic "general discussion" topic, with organic traffic on a site with more Google Juice. IMO he's viewing it through the lens of "something bad happened, we can fix it; bad things shouldn't happen no matter what."

    One of these devs is not like the other one, one of these devs does not belong


  • Banned

    @flabdablet said:

    One of these devs is not like the other one, one of these devs does not belong

    If we start pitting me against my co-founder here into some sort of trolly cat-fight I am going to have to take a break.

    I am using this topic to give the community updates about what I am doing to help cut out the very horrible experience people are having here with forum reliability and performance.

    I do this because we truly appreciate the use/abuse Discourse gets here and want to fix bugs. We need to be careful with our time, this is not a paid gig for us and we are not deployed on Discourse infrastructure. I do this cause I appreciate the forum, but any @sam vs @codinghorror shit needs to die.

    t/1000 is taking 1.2 seconds, 630ms of those are in SELECT "posts"."id" FROM "posts" WHERE "posts"."topic_id" = $1 ORDER BY "posts"."sort_order" ASC . This is way above what I find acceptable but not the only reason we were seeing 500s here. I am still determining all the root causes (as the updates to this topic show)



  • @sam said:

    any @sam vs @codinghorror shit needs to die

    I think you will find that the shit in endless supply around these parts is all WTDWTF vs @codinghorror; there is virtually no WTDWTF vs @sam.

    If you need that shit to settle to the bottom of the tank, your best bet is to gag and handcuff @codinghorror every time you see him heading out to a dicksauce instance with the shit stirring stick in his hand.


  • FoxDev

    @sam said:

    CPU was pegged cause there was a 4 way rebake going for all the posts a certain user here who changed avatars made

    why do i get the feeling that user was me?

    let's see.... i'm one of the highest post count users here... check... i change my avatar frequently.... check...

    damnit! not again! :-P


  • Winner of the 2016 Presidential Election

    Now, now, there is also @RaceProUK.

    Aren't your Avatar-change-times pretty much the same?

    Filed Under: You wouldn't have forgotten about her, would you?


  • FoxDev

    did you accidentally part of your post there?

    ;-)

    @Kuro said:

    Aren't your Avatar-change-times pretty much the same?

    we do coordinate them, yes. :-D


  • FoxDev

    Hedgehog post count: 9840
    Fox post count: 18998


  • Winner of the 2016 Presidential Election

    I typed @race pressed Tab before the popup opened, then pressed enter AS the popup opened but the focus was still on the [←Reply] Button. And pressing X on the "saving" dialoge doesn't stop it...
    So there is that!

    Filed Under: Discourse works in mysterious ways!

    @RaceProUK said:

    Hedgehog post count: 9840
    Fox post count: 18998

    K with tophat and monocle post count: 1525

    Filed Under: Then again,I rarely change my avatar, too. So I think I am in the clear here!



  • Why don't you fix the design in the first place so you don't have to rebake? Just give every user a "virtual" avatar URL, and redirect it to the correct image on the back-end. This is trivial shit. A 3rd grader could come up with this.



  • @flabdablet said:

    How many times was Jeff told that infiniscroll is a non-solution looking for a problem to solve? Not enough times. That's how many.

    Whether he was told one time or 57 million times, it'd never sink in. He has a defensive armor plating of wildly-exaggerated ego that deflects all outside opinion.

    @flabdablet said:

    One of these devs is not like the other one, one of these devs does not belong

    They're both delivering Discourse; they're both equally bad at software development. The only difference is Sam is slightly less of a jerk.

    @sam said:

    I do this cause I appreciate the forum, but any @sam vs @codinghorror shit needs to die.

    You know what would solve that problem? Killing the Discourse product entirely.

    @flabdablet said:

    If you need that shit to settle to the bottom of the tank, your best bet is to gag and handcuff @codinghorror every time you see him heading out to a dicksauce instance with the shit stirring stick in his hand.

    Seconded. Atwood spend months making enemies here.



  • @blakeyrat said:

    Why don't you fix the design in the first place so you don't have to rebake? Just give every user a "virtual" avatar URL, and redirect it to the correct image on the back-end.

    In other words, bake the avatars instead of the posts that reference them. So e.g. instead of /user_avatar/what.thedailywtf.com/flabdablet/120/18181.png,
    .../flabdablet/45/18181.png, .../flabdablet/32/18181.png and .../flabdablet/20/18181.png, I'd have /user_avatar/what.thedailywtf.com/flabdablet/120.png, 45.png, 32.png and 20.png.

    HTTP has a bunch of stuff built into it for cache control. Why not just use that properly instead of relying on a change of avatar filename to force avatar invalidation on change?

    It's not rocket science.
    http://www.youtube.com/watch?v=1xUMow5rhHM&index=10&list=PL565C42B8343143BD


  • FoxDev

    @flabdablet said:

    In other words, bake the avatars instead of the posts that reference them. So e.g. instead of /user_avatar/what.thedailywtf.com/flabdablet/120/18181.png, .../flabdablet/45/18181.png, .../flabdablet/32/18181.png and .../flabdablet/20/18181.png, I'd have /user_avatar/what.thedailywtf.com/flabdablet/120.png, 45.png, 32.png and 20.png.

    Thing is, this is kinda sorta more-or-less how it's done in a way after a fashion if you loosen your definitions a little…



  • @flabdablet said:

    In other words, bake the avatars instead of the posts that reference them. So e.g. instead of /user_avatar/what.thedailywtf.com/flabdablet/120/18181.png, .../flabdablet/45/18181.png, .../flabdablet/32/18181.png and .../flabdablet/20/18181.png, I'd have /user_avatar/what.thedailywtf.com/flabdablet/120.png, 45.png, 32.png and 20.png.

    Huh?

    I'm going in here not actually sure of why "baking" is a thing that needs to happen as a separate process from serving-up the content.

    But I was thinking more like WHAT EVERY OTHER GODDAMNED FORUM PRODUCT ON EARTH DOES having the URL be something like what.thedailywtf.com/avatars/blakeyrat_large.png and just making a copy of the new avatar when it's uploaded and placing it in that path.

    But I guess that's not convoluted enough for a Ruby programmer to figure out. You need 57-60 steps for everything involving at least 3 separate buggy, incomplete open source libraries with trendy names like "Degauss".



  • @blakeyrat said:

    making a copy of the new avatar when it's uploaded and placing it in that path.

    That "making a slightly transformed copy" thing? That appears to be what "baking" means around these parts.


  • FoxDev

    The problem is the avatar is uploaded to a new URL instead of overwriting the old one; the old ones do sometimes get redirected to the new URL, but the way it's done can mean browsers just don't pick up on it. Plus, even if they did, some avatar sizes are not updated at all.



  • @flabdablet said:

    That "making a slightly transformed copy" thing? That appears to be what "baking" means around these parts.

    1. That's not what they're doing (check your avatar image URL before and after changing it)

    2. If that's what "rebaking" is supposed to be doing, it's really really broken.

    But again, I don't get the point of "rebaking" at all. Why don't you just add a flag in the DB table that says, "needs rebaked" and have the server do it at the time it serves that content up? Instead of running some weird memory-hogging background task? I mean, the server already knows how to "bake" stuff obviously, since it does it when people post.

    (Of course then you reach the obvious conclusion: why have the concept of "baking" at all? Just have the damned server do it every time! But that's probably too advanced.)

    Oh because my solution is far too simple, right.



  • @Kuro said:

    Discourse worksfails in mysterious ways!

    FTFY



  • @RaceProUK said:

    The problem is the avatar is uploaded to a new URL instead of overwriting the old one; the old ones do sometimes get redirected to the new URL, but the way it's done can mean browsers just don't pick up on it. Plus, even if they did, some avatar sizes are not updated at all.

    Wat? :facepalm:


  • BINNED

    @sam said:

    I appreciate the forum, but any @sam vs @codinghorror shit needs to die.

    The only conflict I can discern following this line of thinking is that the people here enjoy interacting with you more. And I wouldn't say that's only because you fix shit for us: you can take a joke, run with it and have fun. Hell, you even posted a WTF of your own on here.

    Look, I don't think anyone here garners any genuine hate towards you or the rest of the Discourse team. We get pissed. We yell and rave. And there are some personas on the forum that will sling shit your (collective "your") way all the time. But I am sure, with (spitballing) 90% confidence, that all of us could probably sit down, have a beer, and laugh about this shit IRL.

    I don't think @codinghorror is a person who's good at communicating with this community. That's OK, not everyone is. What got some of us here angry is his insistence it's the fault of the community as a whole, not an individual who can't fit in. I think we may have genuinely riled him up again today. If you're reading this Jeff, we're just having a laugh man. Chill out.

    And yes, we'll tell jokes about how you should kick your boss' ass at times. But that's it. A joke. We insult each other all the time. I call people here "bastards" when I want to express genuine appreciation for having them around.

    Now chill out, pour yourself a fresh cup of Jeff's blood and get the fuck out of here. I'm serious. Go enjoy your vacation, you lucky bastard.

    Now I should really go and get to work. Unlike some people in here...


  • ♿ (Parody)

    @blakeyrat said:

    2) If that's what "rebaking" is supposed to be doing, it's really really broken.

    Baking and rebaking mean to take the raw markdown and generate the html that the browser renders.

    @blakeyrat said:

    Of course then you reach the obvious conclusion: why have the concept of "baking" at all? Just have the damned server do it every time! But that's probably too advanced.)

    It presumably saves time to render the cached (i.e., semi-static) html than to generate it each time.



  • @blakeyrat said:

    check your avatar image URL before and after changing it

    My avatar before changing:

    Size 20: /user_avatar/what.thedailywtf.com/flabdablet/20/18181.png
    Size 32: /user_avatar/what.thedailywtf.com/flabdablet/20/18181.png
    Size 45: /user_avatar/what.thedailywtf.com/flabdablet/20/18181.png
    Size 120: /user_avatar/what.thedailywtf.com/flabdablet/20/18181.png

    I change it to this:

    Now I have:
    Size 20: /user_avatar/what.thedailywtf.com/flabdablet/20/20265.png
    Size 32: /user_avatar/what.thedailywtf.com/flabdablet/32/20265.png
    Size 45: /user_avatar/what.thedailywtf.com/flabdablet/45/20265.png
    Size 120: /user_avatar/what.thedailywtf.com/flabdablet/120/20265.png

    That is: a bunch of slightly transformed (i.e. cropped, resized and format-converted) or "baked" copies of the picture I uploaded, which also have a completely new name for the sole reason, as far as I can see, of creating the otherwise unnecessary and time-consuming work of "re-baking" all the posts that reference my old avatar in quotes.

    Are we in heated agreement yet? I can't tell.


Log in to reply