The Sad State Of (Atwood's) Web App Deployment



  • This just popped up in my NewsBlur, PROBABLY OF INTEREST TO PEEPS HERE.



  • Wow. Installing my test discoinstance back in the early bot days wasn't quite that hard, but it definitely want just grab an image and go.

    Filled under: first mobile post, how do you people put up with this ui?


  • area_deu

    Does Eevee have an account here? They sure should!


    Filed under: Piss Horse



  • Very interesting read. Author clearly has more patience than I; I would have quit trying to get piss-horse running way, way sooner.

    Also because I first read it on my phone which normally has JS disabled:

    <noscript>
        <p>Apologies, but part of running a static blog is that the comments are served by Disqus's JavaScript slurry.</p>
        <p>If it's any consolation, you're probably not missing much.  :)</p>
    </noscript>
    


  • Yep, stuff like this is basically why I mentally add a '1' in front of any claim about "X minute installation". Seems like I'll have to up that to a '2' soon.

    Seems to me nowadays just about any "web 3.0" application requires you to break your server in horribly imaginative ways to get it running. Oh, and also requires some shiny new "deployment mechanism" (which isn't).
    Is it just me or are 'modern' web developers lately just doing the equivalent of "here, take a tar of my dev folder, you should be able to set that up, right?"

    EDIT: Actually, it's more like "Here's a VM image of my dev workstation, ..."



  • I can relate to this deeply, I've had many similar experiences with various things and the usual solution is to wait a few months and try again when it all magically works. At least biicode is working great for me so far.



  • Fun fact: Installing a Go program from source is 1 command (go get github.com/blakeyrat/my-really-cool-forum) and it generates a single statically-linked executable that can be sent to machines without Go installed. Additionally, you can set an environment variable to get an executable for a different architecture or operating system (GOARCH and GOOS, respectively). This compiles the entire standard library and takes about 1 second longer than a compile with a preinstalled standard library does overall.



  • I'm guessing this is because Go's design was influenced by people who actually have to perform those installations?



  • Go was meant to be used, unlike RubyDiscourse.



  • Mobile website development isn't great either.



  • Rant 1!
    Why do we still have 32 bit nonsense. Can't we just draw a line in the sand now say everything going forward must be 64bit and reasonable attempts must be made to port everything from 32bit to 64bit. I know this will be difficult and some porting jobs won't be worth the effort and should be taken out back and shot but can we just move the fuck on. I have a feeling I'm missing something obvious.

    Rant 2!
    Docker and its surrounding ecosystem is a pain in the bollox. Can we just move to boxfuse instead. Yes I will fucking shill this company for forever and a day or until something better comes along.

    All in all it might not be Discuntorse but just about any ROR software who have bought into the OSS is magical nonsense.



  • Oh wow, it gets even better. It's literally unreadable on my phone:

    The content disappears below the fold, although you can see some snippets popping through.

    Can someone copy-paste the article here for the poor WP user?

    EDIT:
    No need, I read it on my puter.



  • @DogsB said:

    some porting jobs won't be worth the effort

    [It depends][1].
    [1]: http://ithare.com/size-matters-is-64-bit-app-always-better/

    "So, if you need more than 2–4G RAM, or if you have lots of computational stuff, or if ... it performs better with 64-bit – by all means, recompile to 64 bits ... However, there are cases ... where [the] move to 64 bits can make your application slower ..."



  • @ashkante said:

    Actually, it's more like "Here's a VM image of my dev workstation, ..."

    That's basically it... a TL;DR explanation of Docker and stuff.



  • Personal experience form yesterday:

    Trying to install a meteor "app". The official documentation was:
    "Use DigitalOcean"

    And here are the entire contents of the README file:

    1. npm install
    2. node main.js
    

    I've produced better documentation on toilet paper after performing a biological function.



  • Well, even then, it's a lot better than having to go through 20 steps to get a forum software working.


  • Fake News

    While the article is right in a lot of ways that this installation is way too complicated, I do feel like the author went to a whole lot of trouble to get it installed on a special snowflake of a system - that entire thing about compiling Ruby or getting a fresh Docker could have been avoided if shez was using a more modern server distro with a consistent userland.

    Everything after that is still valid though. because everyone tries to do Ruby differently.


  • BINNED

    Heh. I went the other way, tried to actually upgrade Docker and all the stuff required. Also, I'm pretty sure it was outdated in their own image*.

    I installed postgresql-contrib, and then some funny things happened. Long story short, I was running Postgres 9.1, and the current Ubuntu version is 9.3. I’d originally installed the postgresql package, and using Arch Linux on my desktop has spoiled me into thinking that that will keep me on the latest version, but Ubuntu cares about trivialities like “not breaking your entire server” and had just kept me on 9.1 the whole time.

    Unless Ubuntu does something weird with it, the author could've saved some time there. On Debian at least, Postgres will indeed not upgrade your current cluster to a newer version to avoid any potential DB breakage. Instead, it will install the new version alongside your current one and spin it up on a different port. Standard Postgres port is 5432, so he actually might have had, or could have, installed Postgres 9.3 which would then run on port 5433**.

    My experience is admittedly limited here, but as far as I can tell, installing a Rails app is impossible.

    Hear hear! Well, maybe both the author and me are completely incompetent buffoons, but this buffoon managed to install multiple PHP and Python apps no problem. Hell, I installed Flarum "the unsuported way", and that includes bot PHP and Node, the other day.

    Only one thing can bind to port 80 and it has to run as root

    Ok, the second part is debatable but... how, pray tell, the fsck do you intend to bind more stuff to a single port on any OS and have it make any sort of sense?

    Or you can do what most people do and add it to the system’s daemon manager, as root.

    Noes, I cannot mess with system startup as regular Joe!

    Wait, wut?

    So it should come as no surprise that there is no way whatsoever to list dependencies on services. You’d think that a web app could just have some metadata saying “I need Postgres and, optionally, Redis”, but this doesn’t exist.

    To run them or install them? Both Upstart and systemd can define dependencies and wait for other required services to start. Package managers can handle installing the dependencies. Web apps not getting packaged like that is another story. phpMyAdmin and pgMyAdmin (to name the two off the top of my head) manage it, so it's not impossible.

    If this is a shared host and you only have one Postgres database, that’s totally fine, because this is a magical world where people actually know about and use Postgres schemata, and apps actually support them.

    Good luck, Discodevs haven't even figured out foreign keys yet.

    We should have this by now. It should work, it should be pluggable and agnostic, and it should do everything right

    Hold on, hard to type from the floor, especially when rolling.


    * might be fixed now. Hey, I can dream!
    ** assuming there was no 9.2 installed as well. I never saw it hit repos in any case.


  • Discourse touched me in a no-no place

    @Salamander said:

    Very interesting read. Author clearly has more patience than I; I would have quit trying to get piss-horse running way, way sooner.

    Oh, hai!

    https://what.thedailywtf.com/t/so-then-i-decide-to-take-the-plunge-and/663

    Nice to see that 15 months later, people are still having issues...



  • We can all agree discourse is (needlessly or not) very complex. But I'm calling bullshit on this article.

    At this point I was pretty sick of Docker, so I decided to try installing the damn thing manually. It was just a Rails app, after all, and I’ve managed to install those before. How hard could it possibly be?

    So he intentionally decided not to follow instructions, and do things manually. Fine. But then, don't go whining about how long it took you.

    Ha, ha! After a git clone (because the app isn’t in rubygems??), I then spent maybe six hours fighting with RVM.
    ...
    See, I actually have a 64-bit kernel, but a 32-bit userspace. (There’s a perfectly good reason for this.)

    So, rails doesn't know how to handle his special snowflake customized environment. Notice this has nothing to do with Discourse at this point. Other rails things would break too.

    The documentation claims the app needs to be (db) superuser. ... Yes, this definitely seems like something a web forum needs. I opted not to give it root on my entire database, which of course broke the migrations because they use CREATE EXTENSION to load binary extensions into my server, a perfectly reasonable thing for database migrations to be doing.

    It's almost like the system is intended to be used on a private db instance inside a docker container. You know, where it makes sense to have full access to the database and be able to optimize it using custom extensions.

    I installed postgresql-contrib, and then some funny things happened. Long story short, I was running Postgres 9.1, and the current Ubuntu version is 9.3......... whine whine

    Basically, he misconfigured his linux again. Nothing to do with Discourse at all. Also, another thing he wouldn't have to worry about if he just used docker.

    At this point the docs take a wild detour into installing some Ruby process management library called Bluepill ..... Anyway, I opted to not do all that, and just ran the thing directly with rails server.
    ...
    Almost done. Now I just need to proxy nginx to it. .... I decided to hell with it and just proxied the whole thing and I’ll fix it later if I feel like it.
    ...
    Now we’re up and running! Except I never get any signup email, and it turns out this is because I also have to run “sidekiq”, a job processor.

    More examples of: "I'll just half-ass it, fuck it" .... (5 minutes later) ... "WHY DOESN'T IT WORK, DISCOURSE SUUUUCKS!!!"

    Set aside the oddball tool breakage and consider that if you follow the instructions to the letter, this web forum requires:

    • Cloning (not installing!) the software’s source code and modifying it in-place.
    • Copy-pasting hundreds of lines of configuration into nginx, as root, and hoping it doesn’t change when you upgrade.
    • Copy-pasting hundreds of lines of Ruby for the sake of bluepill, and hoping it doesn’t change when you upgrade.
    • Installing non-default Postgres extensions, as root.
    • Running someone else’s arbitrary database commands as a superuser.
    • Installing logrotate configuration, as root.

    ALL OF WHICH WOULD HAVE BEEN FINE IF YOU WENT WITH THE DOCKER SETUP, AS YOU SHOULD HAVE

    I'll also notice he initially tried to claim the high ground by not naming the product ("I don’t want to name and shame here"), but then later directly linked to the Discourse github page in the quote above.

    Classy.

    TLDR: Due to his badly hacked together system, the OP fails to install docker. He then tries to manually hack things together and wastes a lot of time troubleshooting, again mostly due to his stupid customized setup. He whines again and again about "bad practices", disregarding that the whole setup is intended to be run inside a container. Finally, outs the forum software in question, even though he initially claimed he won't do that.

    His final conclusions do have some merit, but I'll say his problems with Discourse are 90% his own fault.



  • @cartman82 said:

    I'll say his problems with Discourse are 90% his own fault.

    And the other 10% are Linux?

    :fishing_pole_and_fish:



  • @ashkante said:

    And here are the entire contents of the README file:

    1. npm install
    2. node main.js

    Don't leave us hanging. Did it work?



  • @Onyx said:

    how, pray tell, the fsck do you intend to bind more stuff to a single port on any OS

    SO_REUSEPORT

    @Onyx said:

    and have it make any sort of sense?
    Uhl... oh look at the time! Gotta go.

    Seriously though, it may make sense if for some reason you want to bind more than one process of the same executable instead of forking, or if you want to boot up a new version of a server before taking down the previous version.


  • area_deu

    @cartman82 said:

    TLDR: Due to his badly hacked together system, the OP fails to install docker. He then tries to manually hack things together and wastes a lot of time troubleshooting, again mostly due to his stupid customized setup. He whines again and again about "bad practices", disregarding that the whole setup is intended to be run inside a container. Finally, outs the forum software in question, even though he initially claimed he won't do that.

    Yeah, that was my impression as well.
    Stupid snowflake environments are THE EXACT SITUATION where Docker makes sense, because the app doesn't have to give a fuck about what you did to your system.



  • @ashkante said:

    "Here's a VM image of my dev workstation, ..."

    It works on my machine!



  • @cartman82 said:

    So he intentionally decided not to follow instructions, and do things manually. Fine. But then, don't go whining about how long it took you.

    So, having an application that has "being open source" as one of its major selling points, but actually attempting to install it from source meets with a "*shrug* well gee, we dunno either, use the damn docker or we don't give a fuck" doesn't strike you as absolutely fucking retarded?

    @cartman82 said:

    Other rails things would break too.

    That says more about Rails than about that guy.

    @cartman82 said:

    It's almost like the system is intended to be used on a private db instance inside a docker container.

    It's a forum software, not a fucking hypervisor, for fuck's sake! Next thing you know they'll demand that it gets a server on its own, and that nobody fucking looks at it while it's running!



  • In an age of movie files measured in gigabytes and storage sold by the terabyte, I do occasionally wonder whether shared dynamic libraries are a concept well past its use-by date.


  • BINNED

    @Zecc said:

    Seriously though, it may make sense if for some reason you want to bind more than one process of the same executable instead of forking, or if you want to boot up a new version of a server before taking down the previous version.

    Special case which makes some sense. But the way this was written in the article makes me think he wants to run multiple web apps on 80... but without nginx? I don't completely understand the complaint tbqh... have a web server for each app? Have nginx, Node and whatever else use the same port, WHAT?

    Besides, I think the main complaint is that you can't set this stuff up without root privileges. Well, sorry, but I wouldn't allow that on my system even if it was default. What if someone manages to take over my web server's user account due to a bug? Do I really want them to be able to mess with config files immediately? True, they might be able to get root shell too, but there's at least some security left there. You do not give write access to config files of any kind to any schmoe, sorry.

    Next up he'll complain he needs elevated privileges to run regedit.exe.



  • @Maciejasjmj said:

    So, having an application that has "being open source" as one of its major selling points, but actually attempting to install it from source meets with a "shrug well gee, we dunno either, use the damn docker or we don't give a fuck" doesn't strike you as absolutely fucking retarded?

    Docker also gives you source. I'm pretty sure developers use that setup too.

    @Maciejasjmj said:

    It's a forum software, not a fucking hypervisor, for fuck's sake! Next thing you know they'll demand that it gets a server on its own, and that nobody fucking looks at it while it's running!

    It's a complex software. It demands a modern hardware and OS support. It says so on the cover.

    Whether such requirements are warranted for a forum software is a different debate altogether. But if OP wanted a low requirements forum they can run on a shared host, it's pretty clear Discourse is not the best pick.



  • Dynamic libraries allow you to rebuild one package when the inevitable security breakage happens.

    With static executables, you're going to end up having to rebuild half the system.



  • If you're using a source-based distro like Gentoo, that's not necessarily problematic. You could even do it in the background while all your existing static executables continue to run undisturbed, then restart affected daemons as needed.



  • Docker

    That's really sad. I had high hopes. As in I had hopes, then I got high.

    ... the README just says “use Docker”.

    I wonder if this is a documentation issue. Then again, 'use docker' is pretty much the way docker is supposed to be designed.

    security is kind of a joke with Docker anyway
    Some digging revealed that Docker just doesn’t exist for 32-bit, even though they say it should work (as evidenced by the existence of a canonical 32-bit Ubuntu package), and they just don’t bother mentioning this in their README or installation docs or shell script that runs as root.

    I really, really hope this is due to the technology being young.

    Like I said, high hopes. As in you have to be high to maintain your hopes while being ahead of the curve. Ballmer Peak and all that.



  • @flabdablet said:

    If you're using a source-based distro like Gentoo, that's not necessarily problematic.
    Yes, but if you actually have some sanity (who runs Gentoo on a server?!), then you end up redownloading every single package if there's a libc bugfix - which uses a shitton of bandwidth for starters.

    @flabdablet said:

    You could even do it in the background while all your existing static executables continue to run undisturbed, then restart affected daemons as needed.
    So your plan is for random apps to be vulnerable for a day or more after issuing emerge --update --deep --newuse --verbose world ?

    Because in my experience, rebuilding a system (especially if it has Qt etc) really takes this long.



  • @ashkante said:

    Yep, stuff like this is basically why I mentally add a '1' in front of any claim about "X minute installation". Seems like I'll have to up that to a '2' soon.

    I'd go for a unit upgrade, e.g. minutes => hours, hours => days, etc...



  • @Zecc said:

    Don't leave us hanging. Did it work?

    C:\web\test>npm install
    'npm' is not recognized as an internal or external command,
    operable program or batch file.
    
    C:\web\test>node main.js
    'node' is not recognized as an internal or external command,
    operable program or batch file.
    

    Hmm, can't wrap my head around what the problem might be. The documentation doesn't say anything about it not running under Windows.


  • I survived the hour long Uno hand

    I'm in the camp that this guy is mostly TR :wtf:

    The number of justifiable reasons to run a 32-bit userspace at this point are actually smaller than His High Exaltedness's PQ. And I can't even imagine the logic that goes into justifying whatever gymnastics he had to do to get a 32-bit userspace running on a 64-bit kernel.

    (though, it would be funny to get an Official Support Decree in regards to this guy's experience. assuming the thread stayed visible and didn't just get MegaJeffed).



  • @Minimaul said:

    in my experience, rebuilding a system (especially if it has Qt etc) really takes this long.

    If you were actually going to get serious about doing this (which I never have) you'd probably do an update script that prioritized your long-running daemons and restarted them as it went.

    As I said, this is something I've often idly wondered about, not something I've seriously considered actually doing. Might be fun to play with a system set up this way at some stage and see how responsive it feels, though - statically linked executables do tend to start up lightning-quick compared to dynamically linked versions of the same things.



  • @izzion said:

    The number of justifiable reasons to run a 32-bit userspace at this point are actually smaller than His High Exaltedness's PQ. And I can't even imagine the logic that goes into justifying whatever gymnastics he had to do to get a 32-bit userspace running on a 64-bit kernel.

    Personally, I think he has the expertise required to make it run (or so it seems), so I don't mind that he tried doing that. His beef was mostly with how poorly documented these things can be and how little thought developers have given to anyone outside of the narrow "easy guide".

    And I agree with him that some techniques of modern app deployment are a little iffy, especially piping web output into a shell.

    The over-dramatization is just a communication method. (It seems to be working, judging by all the discussion around this) :smile:


  • sockdevs

    @Weng said:

    I wonder what forum software this guy is talking about.



  • @Weng said:

    I wonder what forum software this guy is talking about.

    Let’s just say it rhymes with “piss horse”.

    Hmm... nope, still can't figure out which one.


  • BINNED



  • See, I actually have a 64-bit kernel, but a 32-bit userspace. (There’s a perfectly good reason for this.)

    Doubtful. I think the author might be TRWTF.


  • Discourse touched me in a no-no place

    See, you'd think Discourse would try to prevent this kind of thing.



  • @Keith said:

    Can someone copy-paste the article here for the poor WP user?
    You can tap the book icon in the Location Bar. It almost never works except on a WordPress site, which this is.@Onyx said:
    Ok, the second part is debatable but... how, pray tell, the fsck do you intend to bind more stuff to a single port on any OS and have it make any sort of sense?
    On Windows, if you have it set up this way, the kernel binds to port 80 and reads the headers, discovering which hostname and path were requested. It then hands it off to the app that bound to that port, hostname, and path. Also, if you do it that way, you're a moron, because HTTP.SYS has more holes than a freshly manufactured LED.@Shoreline said:
    I really, really hope this is due to the technology being young.
    I know version numbers are meaningless (see: Chrome and Firefox "Every Six Weeks"), but it is on 1.8 now. As in post-1.0.

    @ashkante said:

    ```
    C:\web\test>npm install
    'npm' is not recognized as an internal or external command,
    operable program or batch file.

    C:\web\test>node main.js
    'node' is not recognized as an internal or external command,
    operable program or batch file.

    C:\Users\dev_07>npm -v
    1.4.28

    C:\Users\dev_07>node -v
    v0.10.33

    <abbr title="Works On My (out of date) Machine">WOMM</abbr>

  • BINNED

    @TwelveBaud said:

    On Windows, if you have it set up this way, the kernel binds to port 80 and reads the headers, discovering which hostname and path were requested. It then hands it off to the app that bound to that port, hostname, and path.

    Right. Which is exactly what nginx would do, too.

    But I'm sure you can do that on Windows without elevated privileges, so that's OK.


  • Grade A Premium Asshole

    @flabdablet said:

    If you're using a source-based distro like Gentoo

    We are having a discussion about shit being impossible to install, and you casually throw Gentoo in to the conversation? Either you are trolling, or you have a neckbeard large enough to be home to a family of Peregrine falcons.


  • Discourse touched me in a no-no place

    I don't remember the last time I wrote Windows code that used shared libraries beyond the base OS and .net framework (and with a grand total of 3 of those to from, all of which install in parallel with no issues and in fact are wrapped up in the same installer).

    If I need a library, it goes into the application package.

    As I said elsewhere, Windows devs gave up on library hell around the time we figured out it made administration impossible and disk space became cheeeeeap.


  • Grade A Premium Asshole

    @Minimaul said:

    Yes, but if you actually have some sanity (who runs Gentoo on a server?!),



  • @DogsB said:

    Rant 2!Docker and its surrounding ecosystem is a pain in the bollox. Can we just move to boxfuse instead. Yes I will fucking shill this company for forever and a day or until something better comes along.

    I haven't tried to use docker for anything, and I know this guy's pain when you need a version that isn't in the package manager. Also, I thought the discodevs said discourse was 64-bit only. ISTR @PJH having trouble with a 32-bit instance.

    But...they obviously have gone to a lot of work to make stuff work within docker. Of course it's going to be a lot harder to install when you do it manually.

    Of course, dependency management is a PITA. It would be nice if someone came up with a universal solution. Also, Coke in the drinking fountains.


  • Discourse touched me in a no-no place

    Yes. Dependencies are hard. But in Windows we make it hard for developers. On Linux (and crummy Linux ports) we make it hard for users and administrators.


Log in to reply
 

Looks like your connection to What the Daily WTF? was lost, please wait while we try to reconnect.