Non-deterministic Dependency Resolution



  • What is a must-have feature of a software development platform? Non-deterministic builds!

    it is likely that you will run into a situation where your local node_modules directory will differ from both your coworker's node_modules directories, as well as the node_modules directories on your staging, testing, or production servers.

    I'm starting to understand why y'all use NodeJS: spontaneity. Not only is there the unpredictability of using Javascript, but the joy that's felt in typing nondejs build (or whatever other unholy commandline incantation is used) is like an adventure each time.

    I need more of this in my life.


  • Impossible Mission Players - A

    Feature Request: Non-deterministic post assignment to threads :trolleybus:



  • My least favorite part of that:

    @apapadimoulis said in Non-deterministic Dependency Resolution:

    it is likely

    0_1505934146690_962422e8-d859-4210-947d-e1dc5513e50a-6247963+_b0f76c44e8d4aa2a1b1ee3a4b12997a0.jpg



  • @apapadimoulis I'm working with a project right now set up by a co-worker that uses Nuget, Bower and NPM all in one project. Also MSBuild, Gulp, and whatever the fuck code reads package.json I don't even know the name of.

    It's a cavalcade of shit.


  • SockDev

    @blakeyrat said in Non-deterministic Dependency Resolution:

    and whatever the fuck code reads package.json

    That's what NPM reads.


  • And then the murders began.

    @blakeyrat said in Non-deterministic Dependency Resolution:

    @apapadimoulis I'm working with a project right now set up by a co-worker that uses Nuget, Bower and NPM all in one project. Also MSBuild, Gulp, and whatever the fuck code reads package.json I don't even know the name of.

    NPM is what reads package.json.

    It's not your coworker's fault - that's what the current MVC template in VS ships with. :(



  • @unperverted-vixen said in Non-deterministic Dependency Resolution:

    @blakeyrat said in Non-deterministic Dependency Resolution:

    @apapadimoulis I'm working with a project right now set up by a co-worker that uses Nuget, Bower and NPM all in one project. Also MSBuild, Gulp, and whatever the fuck code reads package.json I don't even know the name of.

    NPM is what reads package.json.

    It's not your coworker's fault - that's what the current MVC template in VS ships with. :(

    Am I the only one who misses Silverlight?



  • @pie_flavor said in Non-deterministic Dependency Resolution:

    Am I the only one who misses Silverlight?

    Yes :tropical_drink:



  • @raceprouk Oh I knew that.

    I meant to type "bundleconfig.json"

    As an added bonus, ONE of these things crashes Visual Studio 2017 like fucking clockwork every 12-15 minutes, so as you can imagine working on this project is heaven on earth.


  • SockDev

    @blakeyrat said in Non-deterministic Dependency Resolution:

    @raceprouk Oh I knew that.
    I meant to type "bundleconfig.json"

    Ah, I've not run across that one yet.



  • Pedantic note: virtually all software is non-deterministic to some extent. Complete determinism is very hard to achieve.



  • @raceprouk It's just a list of which files should get their JS minimized but I have no idea what tool even reads it. I mean the "build" button works so I guess I don't care.



  • @blakeyrat said in Non-deterministic Dependency Resolution:

    I meant to type "bundleconfig.json"

    Bundler and Minifier, a part of Web Essentials. Brought to you by the same extension developer as Farticus.



  • Why would you care? Think of the node_modules dir as a cache. You never touch stuff in there and you don't care how it's managed.

    You can reliably get the same dependency tree by removing your node_modules directory and running npm install whenever you make a change to your package.json.

    Wasn't that hard, was it?



  • @twelvebaud said in Non-deterministic Dependency Resolution:

    Brought to you by the same extension developer as Farticus.

    Oh good. The world needed that.


  • SockDev

    @gleemonk said in Non-deterministic Dependency Resolution:

    Think of the node_modules dir as a cache.

    A cache that is never automatically refreshed unless you explicitly work it into your build pipeline. At least with Visual Studio and NuGet, you only need to check one checkbox, and it'll automatically get the right packages on every build.



  • @pie_flavor No. I actually really wish the concept had gone forward.



  • @magus Silverlight was presented as the Flash killer

    Since Flash is dead, there is no point in Silverlight anymore :tropical_drink:



  • @twelvebaud The former being pretty much required to use the MVC template VS ships with, because when you publish, it uses the min version of your CSS, and no one wants to run the horrible (and slow!) node mess they expect you to use to generate that.

    With the added advantage being that you never touch node and it's awful related tools.



  • @timebandit Everyone always called it a Flash killer. It did a lot more than that, by essentially allowing a sane sort of markup language and actually good programming languages to be used for truly rich web apps.

    Flash couldn't do much of what Silverlight could do.



  • @magus said in Non-deterministic Dependency Resolution:

    essentially allowing a sane sort of markup language and actually good programming languages to be used for truly rich web apps

    What's wrong with HTML5 and Javascript?

    *ducks*



  • @anotherusername They're a just barely adequate display mechanism.



  • @anotherusername said in Non-deterministic Dependency Resolution:

    @magus said in Non-deterministic Dependency Resolution:

    essentially allowing a sane sort of markup language and actually good programming languages to be used for truly rich web apps

    What's wrong with HTML5 and Javascript?

    *ducks*

    HTML5 isn't pure shit.

    Javascript is, however. Can't wait for WebASM to become more widely adopted.


  • SockDev

    @pie_flavor said in Non-deterministic Dependency Resolution:

    HTML5 isn't pure shit.
    Javascript is, however.

    Only if you don't know how to use it properly :P

    Unfortunately, few do :crying_cat_face:



  • @raceprouk You mean, "as little as possible, for little interactive bits, like it was designed for"? Because I definitely agree with that if so.

    HTML is pretty bad by itself, though. It's a markup language with inconsistent positioning rules that don't support obvious things like centering things within other things.



  • @blakeyrat said in Non-deterministic Dependency Resolution:

    @raceprouk Oh I knew that.

    I meant to type "bundleconfig.json"

    As an added bonus, ONE of these things crashes Visual Studio 2017 like fucking clockwork every 12-15 minutes, so as you can imagine working on this project is heaven on earth.

    Yeah. My adventure with the new hipster MS tech ended with two hours of figuring out which magic combination of packages spanning all three or four package managers is going to let me write Angular 2 in TypeScript without the build failing due to missing type hints.

    Then I got the project into a state where the build would hang Visual Studio without as much as a log message and gave up.



  • @maciejasjmj said in Non-deterministic Dependency Resolution:

    Then I got the project into a state where the build would hang Visual Studio without as much as a log message and gave up.

    I think we're Visual Studio hangs when building Angular app buddies! High five!



  • @magus said in Non-deterministic Dependency Resolution:

    @timebandit Everyone always called it a Flash killer. It did a lot more than that, by essentially allowing a sane sort of markup language and actually good programming languages to be used for truly rich web apps.

    Flash couldn't do much of what Silverlight could do.

    I remember when it was called Avalon Everywhere, which directly reflected that you could do almost anything in .NET in multiple browsers, plus have it look just as good as Flash as a side benefit.

    Then the skateboard ad came out and nobody knew what the hell it was good for, so it was just "here's something to build video players in; it has more DRM ability."

    @anotherusername said in Non-deterministic Dependency Resolution:

    What's wrong with HTML5 and Javascript?

    While JavaScript was decent, there was no canvas or HTML5, jQuery was only version 1.0.4 and pretty much only good enough for harmonizing querySelectorAll, and you had to drive XHR around like your own personal elephant (if the browser even supported it). This was also long before JSON was popular, and when identity federation in browsers was "public cookie" (i.e. trash). Flash and Silverlight had rich drawing and media APIs, and Silverlight additionally had a lot of pipe-building goop (XML serialization and deserialization and cryptography) that the rest of the Web platform just didn't. Java had the goop too, but it was starting to gain its reputation for being insecure and icky and it just wasn't suited to the kind of flashy things that companies wanted.

    Now HTML5 + JS has all of those things (except decent crypto).



  • Seeing that you don't have much choice in client side web technology besides HTML5 and JavaScript, I've found typescript to make it at least less painful.



  • @the_quiet_one said in Non-deterministic Dependency Resolution:

    Seeing that you don't have much choice in client side web technology besides HTML5 and JavaScript, I've found typescript to make it at least less painful.

    Well, you can always use Emscripten, or one of the zillion languages which compile directly to JS (my favorite being Kotlin).



  • @pie_flavor said in Non-deterministic Dependency Resolution:

    @anotherusername said in Non-deterministic Dependency Resolution:

    @magus said in Non-deterministic Dependency Resolution:

    essentially allowing a sane sort of markup language and actually good programming languages to be used for truly rich web apps

    What's wrong with HTML5 and Javascript?

    *ducks*

    HTML5 isn't pure shit.

    Javascript is, however. Can't wait for WebASM to become more widely adopted.

    Is this memes:trolleybus:?



  • @the_quiet_one The problem with Typsecript is essentially the problem with every JavaScript framework: they only make sense if you're building a whole clientside app in JavaScript.

    I'm of the opinion that JavaScript, even now, should not be anywhere but where it was originally designed to go: random buttons and things, maybe some validation on forms... and as much as possible should be done in a sane language on the server.

    Yes, people older than me are very much on the startup-style javascript hype-train, and want everything to be in the stuff. They are all wrong.



  • @magus said in Non-deterministic Dependency Resolution:

    @raceprouk You mean, "as little as possible, for little interactive bits, like it was designed for"? Because I definitely agree with that if so.

    From the manual of HSC:

    As HTML has turned out to be incredibly useless very soon, many people have written several extension trying to reduce this uselessness. Most of them only increased it. However, this will shortly comment on some of these extensions, and how you can use them together with hsc. [...]
    JavaScript has been developed to show scrolling messages on your pages. It should have been named ScrollerScript, but as it came out more or less at the same time when the hype on a programming language called Java (poor man's SmallTalk raped by C++ which is expected to solve all problems of the world, including traffic, pollution and AIDS) started, its creators decided to use the term "Java'' in it's name, too.



  • @laoc said in Non-deterministic Dependency Resolution:

    @magus said in Non-deterministic Dependency Resolution:

    @raceprouk You mean, "as little as possible, for little interactive bits, like it was designed for"? Because I definitely agree with that if so.

    From the manual of HSC:

    As HTML has turned out to be incredibly useless very soon, many people have written several extension trying to reduce this uselessness. Most of them only increased it. However, this will shortly comment on some of these extensions, and how you can use them together with hsc. [...]
    JavaScript has been developed to show scrolling messages on your pages. It should have been named ScrollerScript, but as it came out more or less at the same time when the hype on a programming language called Java (poor man's SmallTalk raped by C++ which is expected to solve all problems of the world, including traffic, pollution and AIDS) started, its creators decided to use the term "Java'' in it's name, too.

    Can you link this? It sounds like a fun read.



  • @pie_flavor said in Non-deterministic Dependency Resolution:

    [hsc]

    Can you link this? It sounds like a fun read.

    It is!
    It's a fun tool but apart from it being ancient there are some other WTFs, too. That macro language is actually Turing complete so if you abuse it enough you can create programs out of an unreadable mess of quoting. IIRC the macro library has a "foreach" defined just in terms of macro expansion somewhere, and stuff like "call this Perl script and mangle its output using regexps" :p



  • @LaoC

    The Personal Home Page Tool/Form Interpreter (PHP/FI) started as a messy Perl-script acting as some simple CGI-wrapper. As time passed, it grew to a inflated and inconsistent moloch providing a simple programming language inside HTML-documents. It also supports things like embedded SQL, and some people with other presuicidal syndromes even use it as a plug-in with their w3-server.

    The syntax does a very good job on combining the drawbacks of C with the disadvantages of Perl, in general resulting in a completely unreadable sequence of seemingly random characters. Therefore, it became quite popular among Unix-fossils.

    This is how documentation needs to be written nowadays.



  • @pie_flavor said in Non-deterministic Dependency Resolution:

    The syntax does a very good job on combining the drawbacks of C with the disadvantages of Perl, in general resulting in a completely unreadable sequence of seemingly random characters. Therefore, it became quite popular among Unix-fossils.

    This is how documentation needs to be written nowadays.

    ++
    And keeping with Radia Perlman's "Every algorithm should have an algorhyme", the idea of having a motto song quote for every manpage is also quite cool. Especially those in Austrian (the text for the "copyright" page says something like "How much for a look, till up there where you pee? Kiss my ass, says she, then you'll see it for free")



  • Q: What are jerks?
    A: Jerks are persons suffering by a well-known epidemic called Netscapism[1]. Their sources contain sections like

    <BODY BGCOLOR=#123456>

    <BLINK>Click here!</BLINK>

    But, not all jerks became jerks because they like to be jerks. Very often, jerks are forced to use code as seen above by their employer. In this case, they should be refered to as ``prostitutes''.

    (1) Or that's what it was called in the Dark Ages of the Web when Agi first wrote this. Now, in the Even Darker Though Truecolor Ages, it's mostly a similar disease called Exploritis. However, owing to HSC's lack of colorful buttons, ``wizards'' and the like, HSC users and people suffering from these diseases should be fairly disjunct sets. --mb

    Please note that I'm quite serious about all that. As I don't get paid for every new user, there is no need for me to have as many people as possible using this program. Actually I do not care if you try to use it, find out you can not cope with it, scream out ``Dammit, I'm a loser!'' and jump out of the window from the 27th floor. But I start caring about it as soon as you do not jump out of the window but send me bloody emails with silly questions instead.

    So those who now say ``Fine, I always wanted to learn how to write Makefiles'' should consider looking for some other tool or be prepared for jumping out of the window — just in case.

    If you think you are funny and enable RPLCENT, hsc will still not mind your input. But with great pleasure it will cut all your nice multi-byte characters into decadent western 8-bit ``cripplets'' (note the pun). And your browser will display loads of funny western characters - but not a single funny Japanese one.

    Recently an old western approach to these encodings problems has gained popularity: Unicode - that's the name of the beast - was created as some waste product of the Taligent project around 1988 or so, as far as I recall.

    Initially created as an unpopular gadget not supported by anything, it is now in everybody's mouth, because Java, the language-of-hype, several MS-DOS based operating systems and now - finally - the rotten hypertext language-of-hype support it. At least to some limited extent. (Technical note: Usually you only read of UCS-2 instead of UCS-4 in all those specifications, and maybe some blurred proposals to use UTF-16 later.)

    As hsc is written in the rotten C-language (an American product, by the way), it can not cope with zero-bytes in its input data, and therefore is unable to read data encoded in UCS-4, UTF-16 or (würg, kotz, reiha) UCS-2; it simply will stop after the first zero in the input.

    Because the rotten C-language is so widely used, there are some zero-byte work-around formats for Unicode, most remarkably UTF-8 and UTF-7. These work together with hsc, although with the same limitations you have to care for when using the eastern encodings mentioned earlier. Read: Don't use the option RplcEnt.

    Note that it needs at least five encodings to make Unicode work with most software - again in alphabetical order: UCS-2, UCS-4, UTF-16, UTF-7 and UTF-8. I wonder what the ``Uni'' stands for...

    Once upon a time, HTML-4.0 was released, and it sucked surprisingly less (as far as ``sucks less'' is applicable at all to HTML). Of course there still is no browser capable of displaying all these things, but nevertheless you can use hsc to author for it - with some limitations.

    jfif/jpeg
    Essentially, this format does a good job as an idiot indicator. This already starts with the name: jfif is short for JPEG File Interchange Format (because it uses jpeg compression for image data). Nevertheless jfif-pictures will have a file extension set to .jpg'' or.jpeg''.

    The main feature about jpeg compression is that it is lossy, which means that the output image is not identical to the input image. Documents about software using jpeg compression usually claim that you can obtain good compression levels with no visible change. As long as you do not start to rotate or apply other complex image processing operations on pictures, this might even be true. Otherwise, an ugly grid will appear soon.

    Most important, jfif is commonly used to encode images of more or less undressed people. To store as many such pictures as possible in as little space as possible, the compression rate is usually set to a high value. This makes these people look as if they suffered from leprosy, or just a drawing by Egon Schiele.

    Furthermore many people outside English speaking countries pronounce jpeg (jay-peg'') as GPEG (gee-peg''), even if they are normally capable of a proper English pronunciation.

    Once upon a time, some guys invented ASCII. As these guys were Americans, they did not care (or probably not even know) about the rest of the world and said: ``128 different characters are enough.''

    I love this guy.



  • @anonymous234 said in Non-deterministic Dependency Resolution:

    Pedantic note: virtually all software is non-deterministic to some extent. Complete determinism is very hard to achieve.

    I don't know if it's still true, but it probably is. Visual C++ 6 made reproducible builds (in the sense described in that page) impossible.

    Why?

    Well, the short answer is "timestamps".

    Where are the timestamps? In the various files generated by the compiler and linker(1), notably the .obj files generated by the compiler (showing when the source was compiled), the .lib files generated by the librarian (showing when the .obj files in the library were compiled), and the .exe and .dll files generated by the linker (including the timestamps from the .obj files AND a separate timestamp for when the linker ran).

    Even if everything else about the build is the same, getting the timestamps to match exactly is ... hard ... at the best of times.

    (1) I'm old enough to keep wanting to call it a "linkage editor".


  • Discourse touched me in a no-no place

    @steve_the_cynic What are you doing that requires that level of matching?

    FWIW, high-reproducablity builds in languages like Java and Python depend on specifying exact versions of dependencies and pulling standard builds of those exact versions from a controlled repository. That sort of thing means that you're dealing with precisely known quantities; the bugs that you have are the bugs that you have, whether or not you know that you have them. ;)



  • @anonymous234 said in Non-deterministic Dependency Resolution:

    Pedantic note: virtually all software is non-deterministic to some extent. Complete determinism is very hard to achieve.

    I have never seen complete determinism.... EVER.



  • @steve_the_cynic said in Non-deterministic Dependency Resolution:

    I don't know if it's still true, but it probably is. Visual C++ 6 made reproducible builds (in the sense described in that page) impossible.
    Why?
    Well, the short answer is "timestamps".

    Actually that is part of PE [Portable Execution] format used by all Windows EXE files....I have written multiple compare tools over the years that take this into account [along with issues such as undefined bytes between sections, and other elements]



  • @dkf said in Non-deterministic Dependency Resolution:

    @steve_the_cynic What are you doing that requires that level of matching?

    FWIW, high-reproducablity builds in languages like Java and Python depend on specifying exact versions of dependencies and pulling standard builds of those exact versions from a controlled repository. That sort of thing means that you're dealing with precisely known quantities; the bugs that you have are the bugs that you have, whether or not you know that you have them. ;)

    One of the goals mentioned in the linked page was this:

    • start with the sources for ’gcc’
    • compile them with ’gcc’ and with ’clang’
    • compile them again with the two versions of ’gcc’ that you just built

    The goal is that the outputs of the second round of compilations should be bit for bit identical. Compile and link time-stamps make that hard...

    Also, back when VC6 was an interesting thing to use, our System Test guys would only treat two builds as built the same way from the same sources if they were bit for bit identical.


  • Discourse touched me in a no-no place

    @steve_the_cynic said in Non-deterministic Dependency Resolution:

    The goal is that the outputs of the second round of compilations should be bit for bit identical.

    Ah. Foolishness then.



  • @magus I would like carrots to taste like Skittles but that isn't a possibility until GMO technology reaches scary levels. As a UI guy whose landscape is now predominantly web apps I can't be too picky, so typescript it is.



  • @the_quiet_one you're not adding enough butter and brown sugar.





  • @anotherusername you are correct on this point. It's not Skittles but at least better than the awful taste of underground mole people from the depths which it was it was plucked. Honey helps too.


  • Grade A Premium Asshole

    @pie_flavor said in Non-deterministic Dependency Resolution:

    @unperverted-vixen said in Non-deterministic Dependency Resolution:

    @blakeyrat said in Non-deterministic Dependency Resolution:

    @apapadimoulis I'm working with a project right now set up by a co-worker that uses Nuget, Bower and NPM all in one project. Also MSBuild, Gulp, and whatever the fuck code reads package.json I don't even know the name of.

    NPM is what reads package.json.

    It's not your coworker's fault - that's what the current MVC template in VS ships with. :(

    Am I the only one who misses Silverlight?

    Yes.


  • Grade A Premium Asshole

    @twelvebaud said in Non-deterministic Dependency Resolution:

    Brought to you by the same extension developer as Farticus.

    You have to be fist-fucking me? That is a thing?


Log in to reply
 

Looks like your connection to What the Daily WTF? was lost, please wait while we try to reconnect.