Why are we not rolling back?


  • Winner of the 2016 Presidential Election

    So... the scrolling issue is existant. It breaks the reading experience of any topic where more than !PAGESIZE! posts are unread. It's dumb and multiple people have complained about it.

    It also was not an issue before we did the last update.

    But from what I know, we have backups. We also probably pull from some kind of repository (a GIT repository).
    So that leads me to the question in the title:
    WHY ARE WE NOT ROLLING BACK[ the forums]?

    Maybe this has been discussed in one of the bigger topics about the forums... but I can't reasonably be asked to read them.

    Filed Under: Work


  • I survived the hour long Uno hand



  • @Kuro Because opensource development methodology :tropical_fish:


  • sockdevs

    @Kuro If I had to guess, I'd say there's two reasons:

    1. The update included a one-way migration script that'd have to be reversed
    2. The mods don't want to lose the much-improved flagging interface

  • Winner of the 2016 Presidential Election

    @RaceProUK said in Why are we not rolling back?:

    The update included a one-way migration script that'd have to be reversed

    We use NoSQL, so there sure as hell were no breaking schema changes. :trolleybus:


  • BINNED

    @asdf said in Why are we not rolling back?:

    We use NoSQL, so there sure as hell were no breaking schema changes. :trolleybus:

    FTFY


  • Winner of the 2016 Presidential Election

    @Yamikuronue said in Why are we not rolling back?:

    @Kuro Ask @ben_lubar

    As you might know, that is my preferred way of doing these things anyway. But I figured for the sake of transparency (and because @ben_lubar is asleep), I would just make this a topic open for discussion

    @wharrgarbl said in Why are we not rolling back?:

    Because opensource development methodology

    I understand how the bug could come to be. But why are WE keeping it? Are we OpenSource :trollface:?

    @RaceProUK said in Why are we not rolling back?:

    The update included a one-way migration script that'd have to be reversed

    @asdf said in Why are we not rolling back?:

    We use NoSQL, so there sure as hell were no breaking schema changes.

    I remember @ben_lubar posting something about doing database conversion on his local test machine. I dunno if we implemented those changes?

    @RaceProUK said in Why are we not rolling back?:

    The mods don't want to lose the much-improved flagging interface

    Since I have been denied access to the moderation side of the forum, I'd say the mods should just suck it up for a bit. While that sounds harsh, I firmly believe that not being able to read forum posts is a bigger :barrier: to using a forum than the mods just ignoring flags for a bit.

    Disclaimer:
    I am not suggesting to never update ever again. I just feel like we should wait until this is fixed before we use the current version of NodeBB...

    Filed Under: Because srsly


  • I survived the hour long Uno hand

    @Kuro said in Why are we not rolling back?:

    I would just make this a topic open for discussion

    Yeah, but you didn't highlight him, so I figured I'd do it for you :)


  • Winner of the 2016 Presidential Election

    @Yamikuronue Appreciated, thanks! :)

    Filed Under: a mod doing work?!? :shock:


  • I survived the hour long Uno hand

    @Kuro
    I agree with the sentiment of this post - this scrolling issue is a Sev1 outage for me. The forums may not be completely down, but a significant proportion of threads are somewhat or completely unenterable on mobile, and a number of threads are also broken on desktop.

    Several garage threads are exceptionally bad (WARNING: GARAGE CONTENT AHEAD):

    I know we've tried disabling iFramely and several other plugins with no improvement. I tested a couple theories here and here and I get consistent replication if I enter the second thread at either the top or the bottom and try to scroll back in the other direction.

    Is this something that only a few people are experiencing? Or are all of the garagers experiencing it, but they just switched to paginated and didn't report the problem?

    Pardon me for the mass-mention, but @trust_level_3 - if you have infiniscroll still enabled, can you please take 5 minutes to test these two replication cases and report back if you did or didn't get a page jump, what OS and browser you're using, and what your Settings > Pagination > Posts Per Page setting is?

    • Go to this post, then scroll down slowly until it tries to load more posts (should be roughly around the 10 commandments post, post number 16)
    • Go to this post, then scroll up slowly until it tries to load more posts (around when post 55 or 56 is mid-screen)

    Edit: And if you did switch from infiniscroll to pagination because of this update, can you please explicitly mention that here, so that @ben_lubar has an accurate idea of how widespread the problem is?


  • Dupa

    @Kuro just use pagination and be done with it.


  • BINNED

    @izzion It's not dead, Jim. Seems to be jumping around pretty vividly, actually.

    0_1494589442946_9dc225e8-1416-4ff7-a841-bb9c1146f42b-image.png

    Vivaldi on Linux, otherwise known as "Not Firefox on Linux" since it's all WebKit anyway...


  • sockdevs

    @izzion said in Why are we not rolling back?:

    Pardon me for the mass-mention, but @trust_level_3 - if you have infiniscroll still enabled, can you please take 5 minutes to test these two replication cases and report back if you did or didn't get a page jump, what OS and browser you're using, and what your Settings > Pagination > Posts Per Page setting is?

    I got it in both cases.

    0_1494589461295_4c34006c-638c-41d0-8c8c-70267d3c2142-image.png
    0_1494589481141_8cd58ba5-ccd3-4bfb-8bc7-8673ddb1c3eb-image.png
    0_1494589497653_2300d3ca-3089-4f5c-94dd-c89327cb9425-image.png


  • I survived the hour long Uno hand

    @izzion My mouse at work has a broken scroll wheel. Using Chrome 58 on Windows 10

    Using arrow keys to scroll:

    • halfway through the 10 Commandments post, it suddenly jumped to this post being at the top of the screen
    • In the second test, I read "into a silly argument with myself" (or something very like) (fake edit: went back and looked agian, it's this post) and then suddenly was reading this post instead.

    0_1494589542765_8832ca76-dd76-4509-b15a-43b435e439cb-image.png


  • Dupa

    @kt_ said in Why are we not rolling back?:

    @Kuro just use pagination and be done with it.

    Seriously. I stopped using infiniscroll when it was broken so badly right after the migration that it rendered forum unusable. I didn't look back.

    Sure, pagination has its own problems, like broken pagination buttons, but you learn to live with it.


  • sockdevs

    @RaceProUK said in Why are we not rolling back?:

    0_1494589481141_8cd58ba5-ccd3-4bfb-8bc7-8673ddb1c3eb-image.png

    0_1494589714923_0f9be23a-bc7e-4d0a-9c06-461e52be3e69-image.png

    And still happening


  • Winner of the 2016 Presidential Election

    @kt_ I know you mean well and all.. but are you for real?

    Look, I understand I could circumvent this problem if I switched from what I assume is the forum default (infinite scroll) to pagination.
    Let's - for the sake of the argument - say I did that:
    Wouldn't that still leave every not logged in user with an unsolvable problem? SEARCH ENGINES. SEARCH ENGINES EVERYWHERE :alien:
    Wouldn't changing pagination behaviour for every user be more work than simply rolling back the changes done to the forum and update once it's been fixed on BenLubars testing server?
    Isn't putting the work onto the users of your platform the WORST POSSIBLE WAY to solve a problem?!

    @Yamikuronue Your browser only jumps once using arrow keys? I get 2 or more jumps on multiple occasions!

    Filed Under: :arrows:


    Edit: [@Arantor showed me the way - k]


  • BINNED

    @izzion said in Why are we not rolling back?:

    Go to this post, then scroll down slowly until it tries to load more posts (should be roughly around the 10 commandments post, post number 16)

    17 -> 29

    @izzion said in Why are we not rolling back?:

    Go to this post, then scroll up slowly until it tries to load more posts (around when post 55 or 56 is mid-screen)

    58 -> 43

    Windows 7, Chrome 58.0.numbers, 20 posts per page


  • I survived the hour long Uno hand

    @Kuro said in Why are we not rolling back?:

    Your browser only jumps once using arrow keys?

    In those test threads, yes.

    To be honest, I've been in a daze for the past week or so, so when threads start jumping around I've been chalking it up to "uh, did I scroll past three pages without reading them again?" and then close the browser and go cry myself to sleep do something else.


  • sockdevs

    @Kuro logged out users only get pagination. It's not even a choice for them because of search engines.


  • sockdevs

    @Yamikuronue sounds like hugs are needed.


  • Winner of the 2016 Presidential Election

    @Arantor Damn, stop backstabbing me from below my post.

    Filed Under: I guess I will retract that point?


  • sockdevs

    @Kuro not an intentional stabbing stab, just a "not sure if you knew this"

    If I were to stab you, I'd be all Mal Reynolds - you'd be facing me, you'd be armed and I'd probably be the one in trouble...


  • sockdevs

    @Arantor said in Why are we not rolling back?:

    @Yamikuronue sounds like hugs are needed.

    That, and some stress relief with @Perverted_Vixen :takei:


  • Winner of the 2016 Presidential Election

    @Arantor I didn't know it. I can hardly even browse this forum without my stylish on. (Which is annoying everytime I enter the forum from a new device)

    Filed Under: Lesson learned: I should arm myself in preparation


  • I survived the hour long Uno hand

    @Arantor Things are fine, it's just a lot of work. I've got a week less time than I planned >.>


  • Dupa

    @Kuro said in Why are we not rolling back?:

    @kt_ I know you mean well and all.. but are you for real?

    Nope. But taking into account how many bugs I've filed for the forum and that none of them got fixed, I don't really see any rollback happening.

    Seriously, using this steaming pile of shit is hard and it's impossible to convince anyone to fix this.

    I mean, the bugs I file are always a result of browsing the forums with the latest Safari for iOS, so a pretty popular product. One of those makes it.impossible to get to the second page of threads in the lounge. They don't get fixed. Because screw me, I suppose.

    Look, I understand I could circumvent this problem if I switched from what I assume is the forum default (infinite scroll) to pagination.
    Let's - for the sake of the argument - say I did that:
    Wouldn't that still leave every not logged in user with an unsolvable problem?
    Wouldn't changing pagination behaviour for every user be more work than simply rolling back the changes done to the forum and update once it's been fixed on BenLubars testing server?
    Isn't putting the work onto the users of your platform the WORST POSSIBLE WAY to solve a problem?!

    Yes, yes and yes and yes and yes, yes, yes and yes.

    So what? This is what happens here all the time.

    I mean, don't get me wrong. I know that both @boomzila and @ben_lubar tend to try and fix some of the bugs and I thank them for it. But most bugs don't get fixed and I hurt every day because of it.


  • sockdevs

    @Yamikuronue I hear that one. Currently on week 1 of 3 of company website rebuild, I actually think it'll need another 3 weeks after for all the last minute changes...



  • @izzion said in Why are we not rolling back?:

    Go to this post, then scroll down slowly until it tries to load more posts (should be roughly around the 10 commandments post, post number 16)
    Go to this post, then scroll up slowly until it tries to load more posts (around when post 55 or 56 is mid-screen)

    Yup, both are broken for me.


  • I survived the hour long Uno hand

    @Yamikuronue @Kuro
    For those test cases, I would only expect a single jump. In the process of building the test cases and watching the DOM in the Chrome Dev Tools, I figured out that NodeBB keeps 40 posts in the DOM once it's done a mid-thread infiniscroll "page update", so the thread isn't long enough to generate a potential bi-directional infinite jump.

    If the test case isn't enough for a NodeDev to prove it's a general thing or a customization specific thing here, I'll be willing to grudgingly slog through making a larger test case and trying to generate an infinijump case. I'm hopeful that those cases plus confirmation that it's generally reproducible is enough to identify the source of the problem?


  • I survived the hour long Uno hand

    @kt_
    I can understand that point of view as well. Thank you for reporting that you switched from infiniscroll to pagination as a result of this update, it will help measure how severe the infiniscroll problem is.


  • Winner of the 2016 Presidential Election

    I just found out that using pageUp anywhere in a topic that has scrolling problems this one for me is to be strongly avoided since it will always trigger some javascript, even if I am not at the edge of the loaded area. sigh

    Filed Under: Is that actually a nodeBB issue or are we using shitty plugins?

    Addendum: It stopped doing it... :wtf:


  • sockdevs

    @Kuro said in Why are we not rolling back?:

    Addendum: It stopped doing it... :wtf:

    Did it for me: it jumped up, then jumped back to the right place.


  • sockdevs

    I switched to pagination, have not observed problems since on either desktop or iOS.



  • @izzion said in Why are we not rolling back?:

    @trust_level_3 - if you have infiniscroll still enabled, can you please take 5 minutes to test these two replication cases and report back if you did or didn't get a page jump

    Firefox 53.0.2 on Windows 7 checking in, with infiniscroll on and 50 posts per (non-existent) page.

    https://what.thedailywtf.com/topic/22899 is unreadable between posts 85 and at least 150

    Scrolled (down) just fine between 80 and 104ish, then jumped down to 120ish. When I scrolled up it jump up to 104ish again. This happened a couple of times as I scrolled up and down.

    Then I scrolled down some more and it jumped from 115ish to 125ish, from 125ish to 135ish, then back up the same intervals when I scrolled up.

    But it's very disconsistent. Sometimes it jumps consistent intervals, sometimes it refuses to scroll, some other times it even jumps down while I'm scrolling up. Might be related to network latency.


  • I survived the hour long Uno hand

    @Zecc
    With the 50 posts per page setting -- I assume you don't get replications on either test case I linked after the TL3 mention (going top down or bottom up in this thread here: https://what.thedailywtf.com/topic/22907/testing-jellypotato-theory-number-2)?


  • I survived the hour long Uno hand

    @izzion I'm observing the same behavior on your test posts as everyone else is with infiniscroll (skip from 17 -> 29 and 58 -> 43).

    Firefox 53.0.2 (32-bit), Windows 10 x64



  • @izzion said in Why are we not rolling back?:

    Go to this post, then scroll down slowly until it tries to load more posts (should be roughly around the 10 commandments post, post number 16)
    Go to this post, then scroll up slowly until it tries to load more posts (around when post 55 or 56 is mid-screen)

    No problems scrolling the whole topic up and down. Though there is a noticeable pause between posts 33 and 34, it doesn't jump.


  • sockdevs

    @izzion Upping the post-per-page lessens the likelihood of jellypotatowarp, but it still happens.



  • @izzion said in Why are we not rolling back?:

    @Zecc
    With the 50 posts per page setting -- I assume you don't get replications on either test case I linked after the TL3 mention (going top down or bottom up in this thread here: https://what.thedailywtf.com/topic/22907/testing-jellypotato-theory-number-2)?

    Accidentally answered in my previous post.



  • Changed pagination settings to 5 per, and refreshed hard :giggity:

    /topic/22907/testing-jellypotato-theory-number-2/ loaded up to post 4 and refused to load more. I had to scroll up and then scroll down faster, at which point it jumped for 4 to 17.

    It then proceeded to load all posts up to 65 without problems (AFAICT), though the post number indicator would occasionally skip a post.


  • sockdevs

    It's possible to set the pagination to 2:
    0_1494593884479_998219e6-349c-4192-80fd-6a8277892d26-image.png

    I hesitate to imagine how unusable that would be...

    (I was trying to set it to 100)



  • @RaceProUK said in Why are we not rolling back?:

    The mods don't want to lose the much-improved flagging interface

    Well, no, but it's not really that big of a deal. We don't get that many flags. Normally. Yesterday was kind of weird.

    I don't know what would happen if we rolled the code back, but I suspect more problems given all of the db conversion that happened.



  • @Kuro said in Why are we not rolling back?:

    I remember @ben_lubar posting something about doing database conversion on his local test machine. I dunno if we implemented those changes?

    Yes. It's how nodebb works. When you update the code you need to run the update script that does whatever data migration it needs to do. If you don't, then it refuses to start up until you do that.


  • sockdevs

    @boomzilla said in Why are we not rolling back?:

    I suspect more problems given all of the db conversion that happened

    That's my primary concern too



  • Would it be too much to say just have everyone's pagination/infiniscroll setting disabled and put it into paged mode until we get the infiniscrolling issues fixed?



  • @anotherusername Yes, because I personally haven't been affected much.



  • @izzion said in Why are we not rolling back?:

    https://what.thedailywtf.com/topic/22899 is unreadable between posts 85 and at least 150. I've given up trying to figure out when it gets better

    I got past those by specifically going to a post (like 85, then 87, then 89...) until I got to the spot it jumped to. Tho another time, I just said fuck-it, and went to the end of the thread without reading.

    @izzion said in Why are we not rolling back?:

    you did or didn't get a page jump,

    Yes

    what OS and browser you're using,

    Win10, current FFx64

    and what your Settings > Pagination > Posts Per Page setting is?

    20


  • area_can

    In conclusion, infinite scroll is :doing_it_wrong: ? I've been using pages for ever, so all the talk of bad scrolling has been really confusing...



  • FYI, I'm upgrading locally (been a long time since that happened, so there's more here than what happened in the latest TDWTF upgrade)...

    Console output...
    ~/nodebb$ ./nodebb upgrade
    1. Bringing base dependencies up to date... OK
    2. Checking installed plugins for updates... OK
    
    A total of 1 package(s) can be upgraded:
      * nodebb-plugin-htmlcleaner (0.1.8 -> 0.1.11)
    
    Proceed with upgrade (y|n)? y
    
    Upgrading packages... OK
    3. Updating NodeBB data store schema...
    [deprecated] `String.prototype.rtrim` is deprecated as of NodeBB v1.5; use `utils.rtrim` instead.
    12/5 08:26:35 [2290] - warn: You have no mongo password setup!
    12/5 08:26:35 [2290] - warn: [cache-buster] could not read cache buster: ENOENT: no such file or directory, open '/home/boomzilla/nodebb/build/cache-buster'
    
    Parsing upgrade scripts... OK | 28 script(s) found
      → [2015/12/23] Chat room hashes...
     skipped
      → [2015/12/15] Upgrading chats...
     skipped
      → [2016/2/25] Social: Post Sharing...
     skipped
      → [2015/12/23] Adding theme to active plugins sorted set...
     skipped
      → [2016/1/14] Creating user best post sorted sets...
     skipped
      → [2016/1/20] Creating users:notvalidated...
     skipped
      → [2016/1/23] Creating Global moderators group...
     skipped
      → [2016/5/28] Giving topics:read privs to any group that was previously allowed to Find & Access Category...
     skipped
      → [2016/4/14] Group title from settings to user profile...
     skipped
      → [2016/6/13] Store upvotes/downvotes separately...
     skipped
      → [2016/4/19] Users post count per tid...
     skipped
      → [2016/4/29] Dismiss flags from deleted topics...
     skipped
      → [2016/8/5] Removing best posts with negative scores...
     skipped
      → [2016/7/12] Giving upload privileges...
     skipped
      → [2016/9/22] Category recent tids...
     skipped
      → [2016/8/7] Granting edit/delete/delete topic on existing categories...
     skipped
      → [2016/10/8] Favourites to Bookmarks...
     skipped
      → [2016/10/14] Sorted sets for post replies...
     skipped
      → [2016/11/22] Update global and user language keys...
     skipped
      → [2016/11/25] Sorted set for pinned topics...
     skipped
      → [2017/2/28] Upgrading config urls to use assets route...
     OK
      → [2017/2/25] Update global and user sound settings...
     OK
      → [2017/4/16] Delete accidentally long-lived sessions...
     OK
      → [2017/4/14] Set default allowed file extensions...
     OK
      → [2017/3/22] Update moderation notes to zset...
      → [2017/3/22] Update moderation notes to zset... OK
      → [2017/2/27] New sorted set posts:votes...
      → [2017/2/27] New sorted set posts:votes... OK
      → [2016/12/7] Migrating flags to new schema...
      → [2016/12/7] Migrating flags to new schema... OK
      → [2017/4/26] Remove relative_path from uploaded profile cover urls...
      → [2017/4/26] Remove relative_path from uploaded profile cover urls... OK
    Upgrade complete!
    
    12/5 08:33:57 [2290] - warn: You have no mongo password setup!
    12/5 08:33:57 [2290] - info: [build] Building javascript
    12/5 08:33:57 [2290] - info: [build] Building client-side CSS
    WARN: Output exceeds 32000 characters
    WARN: Output exceeds 32000 characters
    WARN: Output exceeds 32000 characters
    WARN: Output exceeds 32000 characters
    WARN: Output exceeds 32000 characters
    12/5 08:34:02 [2290] - info: [build] clientCSS => Completed in 4.781s
    12/5 08:34:02 [2290] - info: [build] Building admin control panel CSS
    12/5 08:34:04 [2290] - info: [build] acpCSS => Completed in 2.303s
    12/5 08:34:04 [2290] - info: [build] Building templates
    12/5 08:34:04 [2290] - info: [build] tpl => Completed in 0.197s
    12/5 08:34:04 [2290] - info: [build] Building language files
    12/5 08:34:06 [2290] - info: [build] js => Completed in 9.108s
    12/5 08:34:09 [2290] - info: [build] lang => Completed in 4.836s
    12/5 08:34:09 [2290] - info: [build] Linking sound files
    12/5 08:34:09 [2290] - info: [build] sound => Completed in 0.004s
    12/5 08:34:09 [2290] - info: [build] Asset compilation successful. Completed in 12.235s.
    OK
    
                                                                                     NodeBB Upgrade Complete!
    
    

    This one: [2017/2/27] New sorted set posts:votes... took a looong time.



Looks like your connection to What the Daily WTF? was lost, please wait while we try to reconnect.