The Sad State Of (Atwood's) Web App Deployment



  • @blakeyrat said:

    http://eev.ee/blog/2015/09/17/the-sad-state-of-web-app-deployment/

    This just popped up in my NewsBlur, PROBABLY OF INTEREST TO PEEPS HERE.

    I am sooooooo glad I jumped ship from the L.A.M.P stack about a decade ago to .NET, and haven't looked back since. I recognize everything being said in the article, and I am kind of proud of that, but I am also super-glad I don't have to deal with it any more. The article author is dead-on accurate.


  • Discourse touched me in a no-no place

    @DogsB said:

    Why do we still have 32 bit nonsense. Can't we just draw a line in the sand now say everything going forward must be 64bit and reasonable attempts must be made to port everything from 32bit to 64bit. I know this will be difficult and some porting jobs won't be worth the effort and should be taken out back and shot but can we just move the fuck on. I have a feeling I'm missing something obvious.

    A (I assume) very large library of programs out there that use (say) 32-bit ActiveX controls, for which there is no 64-bit version, is a large part of that.



  • @cartman82 said:

    Umm, yes? How is that different from running a setup.exe you downloaded from the Internet?

    Well, you can look at the code in the script, but not in the exe without a good decompiler.



  • Tarn Adams said he wants to port Dwarf Fortress to 64 bit, but there are probably a lot of bad things that would happen (he calls himself a "bad programmer" and I haven't seen the code) if that happened without a lot of testing and bugfixes.



  • @fbmac said:

    Gobolinux has a very different packaging system, but I didn't try it myself:

    GoboLinux is an alternative Linux distribution which redefines the entire filesystem hierarchy. In GoboLinux you don't need a package database because the filesystem is the database: each program resides in its own directory, such as /Programs/Xorg-Lib/7.4 and /Programs/KDE-Libs/4.2.0. Like it? [Learn more...](http://www.gobolinux.org/?page=at_a_glance)

    TRWTF is that those images at the top-right of the page aren't just for decoration.



  • @ashkante said:

    Oh, and also requires some shiny new "deployment mechanism" (which isn't).

    God forbid anyone use something as feature complete, well understood ans complete as make, that' just so uncool omg you guys...



  • @ben_lubar said:

    Go also supports building as a dynamic library.

    I'm actually learning (and loving) Go at the moment...

    I've not tried this yet. I should.



  • go build -buildmode=shared std
    go build -linkshared path/to/my/command



  • Ben don't share your STDs



  • stdin, stdout.



  • Disclaimer: If you try to find logic in any of my answers here, I claim entrapment. That applies to the previous statement, too.


    @blakeyrat said:

    Ruby worked on it, Nginx did, so did Postgres. How was it "special snowflake?" All the stuff Discourse needed was there.

    @cartman82 said:

    I don't know why ruby doesn't just work with this strange setup. I guess it has to load 32-bit or 64-bit version of some thing depending on system sniffing, and whatever code does that gets confused by the kernel vs userland difference.

    And yes, this definitely feels like it breaks the "OS contract", whatever that is. System should be either 32-bit or 64-bit. If he wants to mess around with custom-compiling his kernel or whatever he was doing, fine, but then you can't expect everything else to work out of the box.

    Several gem native extensions are required, such as nokogiri (xml parser). (I'm guessing 'performance' or 'we're just shelling into libxml'.)

    They come pre-built if you use Docker. They fail to compile if you use the wrong architecture. Take it up with the gem authors.

    @blakeyrat said:

    If Discourse requires Sidekiq, why doesn't it install it itself? Or at least detect if it's running? That's a perfectly valid argument.

    Well, if you use the Docker install, sidekiq gets started by runit.

    If you use the shitty install, sidekiq gets launched by the piece of crap called Bluepill that eevee just decided to skip because it sounded too hard.

    If you do a development install, jobs either are executed immediately or never need to run.

    v
    @boomzilla said:

    You cannot be serious. It does.

    ^

    And if you visit the admin dashboard, it will actually tell you "Sidekiq isn't running, you should go fix that."

    @svieira said:

    Wouldn't it be better to use Ansible or Puppet or some sane configuration management system so if users wanted to make it easy to install Discourse on, say, a RPi (for some reason) they could do so ... in the spirit of Open Source and all that.

    I really don't think you want your machines hooked up to our Puppet server. (How else are you going to update the config when it needs to be? redownloading? okay now you just added ANOTHER top-level wrapper script, good job) But seriously, that's giving someone else a persistent root on your system.


  • :belt_onion:

    Chuckles You are absolutely right - Ansible / Chef Solo / SaltStack local would be the way to go (though I could have sworn Puppet had a local mode). Thanks for the correction!

    My question still stands about Ansible / Chef Solo / Salt Stack to automate setup - why not use it instead of a Docker file? Then you could use Packer and build basically whatever the heck you wanted (Docker , AMI , VMWare image , etc.)




  • :belt_onion:

    @ben_lubar said:

    http://dean.edwards.name/packer/

    Yeah, that was pretty good stuff. I wonder what the size savings are if you gzip a packer-packed file vs. just a normal minified file ...

    Filed under Nah, not worth it



  • Isn't packer basically just ASCII huffman encoding? So it's probably less compressible by gzip. Plus it means your program has to be parsed twice by the JavaScript interpreter.



  • @Vaire said:

    I am sooooooo glad I jumped ship from the L.A.M.P stack about a decade ago to .NET, and haven't looked back since. I recognize everything being said in the article, and I am kind of proud of that, but I am also super-glad I don't have to deal with it any more. The article author is dead-on accurate.

    Try porting one of your .NET apps to Mono, and also, uh, jump back in time a year or two...



  • Because the pipe could break at any moment, causing the command line to execute a very different script than the one you thought you were downloading (in the same way that rm -rf / is a very different command than rm -rf /tmp/$WORKINGDIR).


  • BINNED

    @Buddy said:

    the pipe could break at any moment

    Better call the plumber


  • Fake News

    Good point, though you could also download the script with curl, check and run it.



  • Remember the days when every single open-source download would come with an md5 hash, and the implication that you were a suicidal moron if you didn't verify the download before trying to use it? Now the pendulum has swung in the other direction.

    I wonder where it will swing to next...



  • That would be much easier to deploy in that Ubuntu server in the article



  • @svieira said:

    >We initially tried supporting arbitrary Discourse installs but it quickly became a support nightmare for our small 7 person team

    This is what gets me. BumfuckOS can gather an entire army of lowlives to test, patch, package their no-name Ubuntu clone, but somehow discourse—whose github network is so big GitHub doesn't even try to display it—can't pull together any more than seven contributors?



  • It turns out that how you treat your community somehow affects what kind of contributions you can expect back from that community?


  • Fake News

    Which is why we only contribute memes and telling what the Discodevs are doing wrong.



  • @Maciejasjmj said:

    and that nobody fucking looks at it while it's running!

    Nobody looking at Discourse seems like a damned good idea to me.



  • Exactly.



  • @svieira said:

    >If it helps, I'd be glad to throw in a free Digital Ocean $20/month droplet

    In other words "oh fuck it, I'm gonna have to do it myself using our cargo cult guide because honestly, nobody has any idea how to set it up anymore".



  • @cartman82 said:

    But if OP wanted a low requirements forum they can run on a shared host, it's pretty clear Discourse is not the best pick.

    Discourse is not the best pick, period.


  • Trolleybus Mechanic

    stderr


    Filed under: How sex landed me in the ER



  • @Buddy said:

    Remember the days when every single open-source download would come with an md5 hash, and the implication that you were a suicidal moron if you didn't verify the download

    Too many MD5 collisions these days, I guess...


  • Discourse touched me in a no-no place

    @tar said:

    Too many MD5 collisions these days, I guess...

    Try SHA-512 if you really worry about that.


  • Discourse touched me in a no-no place

    @JBert said:

    Which is why we only contribute memes and telling what the Discodevs are doing wrong.



  • I've seen mostly GPG signatures used nowadays, since anyone who can modify the executable you just downloaded can also modify the hashsum you downloaded from the same server.



  • Linux md5 drivers are assholes.


  • Discourse touched me in a no-no place

    @ben_lubar said:

    I've seen mostly GPG signatures used nowadays, since anyone who can modify the executable you just downloaded can also modify the hashsum you downloaded from the same server.

    You usually serve them from different servers though. The hashes come from the website you're looking at, and the download files (often rather large) come from somewhere that specialises in large file hosting. There are way more options for hosting a website than hosting a gigabyte download…



  • But with the GPG signature, you're verifying that the thing you're downloading is the thing the build machine generated, whereas with a hashsum, you're only verifying that the attacker modified both locations.

    Plus, GPG can be used without HTTPS because HTTPS only provides proof of identity if the server hosting the files is secure, whereas GPG provides proof of identity no matter how many intermediate steps the data had to go through.


  • Discourse touched me in a no-no place

    @ben_lubar said:

    But with the GPG signature, you're verifying that the thing you're downloading is the thing the build machine generated, whereas with a hashsum, you're only verifying that the attacker modified both locations.

    Unless someone's put their GPG key in github… :facepalm: 😄

    @ben_lubar said:

    Plus, GPG can be used without HTTPS because HTTPS only provides proof of identity if the server hosting the files is secure, whereas GPG provides proof of identity no matter how many intermediate steps the data had to go through.

    Strictly, GPG provides message (i.e., file) security, whereas HTTPS provides transaction and connection security.


  • Notification Spam Recipient

    @ben_lubar said:

    But with the GPG signature, you're verifying that the thing you're downloading is the thing the build machine generated, whereas with a hashsum, you're only verifying that the attacker modified both locations.
    What if the build machine has been compromised and the artifact was built with the malicious code.



  • Hopefully the key is not in the build server, but on a separate signing server or the release managers own computer or something. More things to compromise in order to make a successfully compromised release. Probably easier to just sign the release with some other key that has the same user details as the correct key, and hope people won't notice the difference...



  • @DogsB said:

    What if the build machine has been compromised and the artifact was built with the malicious code.

    Well, then you're fucked with either method.

    @calmh said:

    the same user details

    As in the same public key? Not likely.



  • No, but I can generate a new key with the same name and email address and upload it to the key servers. If the downloader doesn't verify the key fingerprint when checking the signature it might seem legit. GPG is a rather user unfriendly piece of software.


  • :belt_onion:

    @blakeyrat said:

    You created a service account

    How can you do that without root? Curious minds want to know.



  • That probably would require root.



  • sudo make sandwich

    <waiting for the blakey to rant on that



  • But...they obviously have gone to a lot of work to make stuff work within docker. Of course it's going to be a lot harder to install when you do it manually.

    Of course, dependency management is a PITA. It would be nice if someone came up with a universal solution. Also, Coke in the drinking fountains.

    Honestly, I'm liking configuration/deployment management scripts like Salt. You use a small DSL to configure all the apps a system needs, and then you run an interpreter on your server to install and configure the system, per the configuration you wrote, parametrized by role or whatever.

    It works very nicely, and makes it easy to coordinate even multi-computer systems, since you just keep the configuration data in your SCM.

    Salt is not 'universal,' and I can imagine a better way to handle it. But it's still good.


  • Discourse touched me in a no-no place

    @calmh said:

    GPG is a rather user unfriendly piece of software.

    Understatement detected. GPG makes me long for the usability of git



  • FWIW, git is the most usable command line tool I've ever used.



  • @DogsB said:

    What if the build machine has been compromised and the artifact was built with the malicious code.

    What if the call was coming from inside the house?



  • @dkf said:

    Understatement detected. GPG makes me long for the usability of git…

    Holy shit yes. It's actually designed to not be scriptable. How much more user-hostile can you get?

    [quote=The Free Software Foundation (current maintainers of GPG)]
    The freedom to run the program as you wish, for any purpose (freedom 0).
    [/quote]


  • Discourse touched me in a no-no place

    @riking said:

    How much more user-hostile can you get?

    I'm not sure whether the OpenSSL suite of tools is better or worse. I am sure that security people should never ever EVERstrong emphasized text be allowed to put an interface in front of a user. They always get it completely wrong.


Log in to reply