WTF NPM
-
I am trying to set up an internal registry for the Unity Packages that my company is using migrating to the new, shiny Unity Package Manager UI. After digging for a while, I found that it was using npm as the backend for the package management.
So I set up an internal npm server for testing, only to find out that the cli commands for npm is ridiculous.
What kind of ecosystem merges account creation and login as the same command.
"npm login is an alias for npm adduser" ????!!!
You have got to be kidding me.
-
I have come to the unfortunate conclusion that npm developers are those who ask for admin access to databases so they can "fix" the database themselves and leave it open for any Tom, Dick, Harry to mess around with databases from code.
-
-
@JBert That license plate should be something npm-related...
-
I have come to the unfortunate conclusion that npm developers are those who ask for admin access to databases so they can "fix" the database themselves and leave it open for any Tom, Dick, Harry to mess around with databases from code.
I thought that's common knowledge.
-
-
I have come to the unfortunate conclusion that
npmdevelopers are those who ask for admin access to databases so they can "fix" the database themselves and leave it open for any Tom, Dick, Harry to mess around with databases from code.FTFY
-
npm is the "God" cli tool that should somehow be responsible for running your own code instead of you running it. It lost me as soon as I saw the first tutorial for a web framework that said you start your app with
npm start. What why the hell should the package manager even... you get the idea.
-
@marczellm said in WTF NPM:
npm is the "God" cli tool that should somehow be responsible for running your own code instead of you running it. It lost me as soon as I saw the first tutorial for a web framework that said you start your app with
npm start. What why the hell should the package manager even... you get the idea.npmis not in bad company in this regard when you see how pretty much everything in the Java world can offer the same (Maven, Gradle, ...). Similarly, .NET Core offersdotnet run.EDIT: Actually, I'm slightly pissed that I'm defending
npmhere.
-
-
Similarly, .NET Core offers dotnet run.
Right, but .net core is the runtime. The equivalent would be
nuget run
-
@Jaloopa I believe the
nugetcommand is deprecated in .NET Core? Or why else would they have inventeddotnet restore?
-
@JBert build tool starting your application is a nice convenience feature - you don't have to hunt down the executable in build directories. The problem with NPM is that
npm startdoesn't start the built package, but rather runs arbitrary script defined in package description.
-
Anyway, reason #638 why I hate npm: it makes packages where the file date / time info is changed to a fixed point in time and then extracts packages using that date / time. This fucks up incremental build tools, incremental deployment tools, you name it - if it looks at the file date / time you're fucked.
Of course, there are several questions. But most of all, why bother changing the file date / time info inside a package?
Apparently they caught a brainworm about "reproducible builds": basically they wanted to build tar/gz packages which have the same cryptographic hash no matter when they are built or on what machine, as long as they are built from the same sources and thus generate the same outputs. And so they set out to get this done: they used a fixed timestamp of
0in their tar/gz packages.However, it turns out that some archival tools treat that as an error (yes, same link as above, just the top of the thread where the issue is described). Oops. Instead of backing off and deciding that maybe they should have just written a proper tool to canonicalize the contents of a package and calculate its hash they chose a different route: they just picked a different, non-zero time (for some it might seem familiar).
So yeah, better get out the workarounds like
touching all the files in the extracted package folder...
-
@JBert build tool starting your application is a nice convenience feature - you don't have to hunt down the executable in build directories. The problem with NPM is that
npm startdoesn't start the built package, but rather runs arbitrary script defined in package description.
Any build tool would still need to figure out what to run and might execute some bit of code in the project descriptor (whether written as a script or declared as a "plugin"). This is not a problem exclusive to npmas for example Gradle files are full-fledged DSL "scripts" which modify the project descriptor. Very flexible, but if you are worried about unwanted code execution then you should audit the whole lot, including any plugins it uses.
-
@JBert
After reading through the responses to issues in their npm.community, I get the feel that npm developers approach to solving problems seems to be just use a different hacky workaround and not actually resolving the actual issue.
-
@WPTI was thinking the same thing.
Another interesting tidbit is that they closed up the Github bug trackers and moved to a Discourse forum instance to do their "bug tracking".
-
@JBert
After reading through the responses to issues in their npm.community, I get the feel that npm developers approach to solving problems seems to be just use a different hacky workaround and not actually resolving the actual issue.As probably every company does when they prefer pushing their products onto the shelf.
also: backwards compatibility
-
Update: Since I realize there is no way of enforcing on user lists, I have implemented an acl on http requests that POST publish/unpublish.
Making the best out of what is available.
-
@marczellm said in WTF NPM:
What why the hell should the package manager even
NPM is “not a package manager”
.(it is rather a “development sandbox manager”).
-
Apparently they caught a brainworm about "reproducible builds": basically they wanted to build tar/gz packages which have the same cryptographic hash no matter when they are built or on what machine, as long as they are built from the same sources and thus generate the same outputs.
That part is fine. And reasonable. The bug is actually this¹:
$ tar --help … -m, --touch don't extract file modified timeOk.
- TIL. I knew Windows default to doing this, but Unix
cpdoes not preserve modified time by default, so I didn't expecttarto do it by default either. 
Ok, that default does make sense for the original purpose of tar, backup and restore, but that requires the other preserve options anyway and for any other purpose it does not make sense.- This means that all the tools that have anything to do with building and packages should actually be calling tar with
-m; it's not justnpm. - But really, build tools should be using file changed time². Because that cannot be fiddled with, and it also reflects changes like renaming, which for a build tool certainly should be taken into account, because if an old file was renamed here, it is still a new content at this location.
¹ However since npm is not using the command-line tar, but rather its own tar library, its authors might have been more aware of the corresponding setting.
² And possibly comparing the newer of changed and modified time of the dependency with the older of changed and modified time of the dependent, since something might accidentally bump the changed time of the dependent and there may be some changes that don't bump the changed time of the dependency. In my testing it didn't happen, but an overwrite without truncate that does not change the length might.
- TIL. I knew Windows default to doing this, but Unix
-
@Bulb Changes to file data do not update change time.
-
@Bulb Changes to file data do not update change time.
Yes, but in practice most changes change the length too and that does update the change time, as does truncating the file and writing back the same content. The correct rule is that the target is out of date if either changed or modified time of the dependency is newer than either modified or changed time of the target. Then tools that fiddle with mtime won't break it (
touchalways changes ctime in my testing).
-
@Bulb Almost all tools that you'd ever want to use in
make(or other build systems) operate primarily on the contents of files; the inode change time is irrelevant to them. If someone decides to usetouchto screw with you, they're probably doing so with a good reason. Or are adult enough to cope with things breaking; that also works as a rule.Tools probably ought to not be using either the file modification time or the inode change time for timestamping builds. They're too fragile. Commit metadata from the VCS is a lot more stable, and much much harder to fake with DVCSs because of the hashing algorithms used.
-
@dkf Our in-house VCS likes reverting files which are unchanged from their base label versions to actual symlinks to the base label on NFS. This plays havoc with
make-based builds if you revert changes in a file.
-
@PleegWat
Every Byte Is Sacred… 
-
@dkf It probably made sense when they built it, which I think was sometime in the nineties.
-
@Bulb Almost all tools that you'd ever want to use in
make(or other build systems) operate primarily on the contents of files; the inode change time is irrelevant to them. If someone decides to usetouchto screw with you, they're probably doing so with a good reason. Or are adult enough to cope with things breaking; that also works as a rule.Tools probably ought to not be using either the file modification time or the inode change time for timestamping builds. They're too fragile. Commit metadata from the VCS is a lot more stable, and much much harder to fake with DVCSs because of the hashing algorithms used.
Using
gitcommit hashes as elements in build identifiers is all very well, but less than totally useful if your DVCS userbase is infested with people who think that history rewrites are a good thing.
-
@PleegWat I'm having PTSD flashbacks right now.
IIRC, Gradle handled it a bit better, since it ignores the modification time and hashes the contents of each file. Horribly slow on NFS, but at least it worked.
-
@Steve_The_Cynic I'll be honest, I'm still on the fence as to whether you should rewrite branch history to reduce the number of commits that show when you merge it into an upstream branch (and/or master).
On the one hand, it makes the history less polluted. On the other hand, you can no longer cherry-pick commits from said branch in case you need to pull them into a third branch.
-
@Steve_The_Cynic said in WTF NPM:
Using git commit hashes as elements in build identifiers is all very well, but less than totally useful if your DVCS userbase is infested with people who think that history rewrites are a good thing.
I think of this as a feature!
-
@powerlord said in WTF NPM:
On the other hand, you can no longer cherry-pick commits from said branch in case you need to pull them into a third branch.
If you merge commits that you'd want to individually apply to other branches during a rebase, then you're combining too much into one commit. The only thing you really want to eliminate are fix-up commits, incomplete end-of-day commits and commits where you tried something that ultimately didn't work. That's clutter you really don't want to see in the history.
Turning every feature branch into huge commits is obviously
.
-
If someone decides to use touch to screw with you, they're probably doing so with a good reason.
No, they are doing it because they don't understand it is actually breaking make. Hell, I've seen several version control systems set date on files to the time they were checked in though they are almost always used with
makeand are breaking it this way.Almost all tools that you'd ever want to use in make (or other build systems) operate primarily on the contents of files; the inode change time is irrelevant to them.
Yes, but no. If the inode changed, it might mean the file was replaced by another that was created earlier, but it still invalidates the build products. Changes to things like permissions are rare in software checkouts, so replacing the file or back-dating it for whatever reason is most likely.
-
@Steve_The_Cynic said in WTF NPM:
Using git commit hashes as elements in build identifiers is all very well, but less than totally useful if your DVCS userbase is infested with people who think that history rewrites are a good thing.
History rewrites are OK on feature branches, but any sane configuration of Git server prohibits rewinding any of the central branches like
master,developetc., so anything with hash that is not found is someone's local build and should not be used for anything anyway.
-
@powerlord said in WTF NPM:
On the one hand, it makes the history less polluted. On the other hand, you can no longer cherry-pick commits from said branch in case you need to pull them into a third branch.
Normally you rewrite so that you can usefully cherry-pick (and more easily review) by reorganizing the changes into logical steps instead of a bunch of back-and-forth experiments as you actually discovered what needs to be done. That's basically the only reason to rewrite.
-
@powerlord said in WTF NPM:
On the one hand, it makes the history less polluted. On the other hand, you can no longer cherry-pick commits from said branch in case you need to pull them into a third branch.
Normally you rewrite so that you can usefully cherry-pick (and more easily review) by reorganizing the changes into logical steps instead of a bunch of back-and-forth experiments as you actually discovered what needs to be done. That's basically the only reason to rewrite.
I'm not joking when I say one project I worked on had a policy where every feature branch had to have its history squashed to a single commit before you did a pull request for it regardless of how complex the change in question was.
-
-
@powerlord said in WTF NPM:
@powerlord said in WTF NPM:
On the one hand, it makes the history less polluted. On the other hand, you can no longer cherry-pick commits from said branch in case you need to pull them into a third branch.
Normally you rewrite so that you can usefully cherry-pick (and more easily review) by reorganizing the changes into logical steps instead of a bunch of back-and-forth experiments as you actually discovered what needs to be done. That's basically the only reason to rewrite.
I'm not joking when I say one project I worked on had a policy where every feature branch had to have its history squashed to a single commit before you did a pull request for it regardless of how complex the change in question was.
I've had a CM that adamantly argued that this is how git is supposed to be used. He kept complaining when I did not do it that way.
-
-
@Carnage Sounds like Git's biggest problem is the idiots that try to use it.
-
@Carnage Sounds like Git's biggest problem is the idiots that try to use it.
git's biggest problem is there's too many ways to use it and most of them are
-
@Carnage Sounds like Git's biggest problem is the idiots that try to use it.
git's biggest problem is there's too many ways to use it and most of them are
I spent quite some time at some point attempting to convince a colleague that being able to maintain significant differences between your own working copy and other people's while at the same time doing unrelated work is, in corporate settings, a misfeature.
-
@Carnage Sounds like Git's biggest problem is the idiots that try to use it.
git's biggest problem is there's too many ways to use it and most of them are
I spent quite some time at some point attempting to convince a colleague that being able to maintain significant differences between your own working copy and other people's while at the same time doing unrelated work is, in corporate settings, a misfeature.
I sort of realized that the use-case for git in the corporate world, is really limited. What usually is desired in corporations is central control of source code, which eventually makes svn the most fitting solution for most corporate use-case instead of git.
-
makes svn the most fitting solution
If it had better IDE integration, fossil would be more suitable.
-
What usually is desired in corporations is central control of source code, which eventually makes svn the most fitting solution for most corporate use-case instead of git.
It would be, if it could do merging with any reliability and had temporary branches.
The thing with distributed version control is that it really matches the process better. The revisions are really created independently in parallel and then merged together, so forcing a linear history and requiring naming branches to create any parallelism is just creating friction. Just notice that subversion (and really every centralized system) has two merge operations—update and merge. They do exactly the same thing, just in different contexts.
You could have the model used by distributed systems with central storage though and in corporate setup it would make sense.
If it had better IDE integration, fossil would be more suitable.
Fossil is just as distributed… hm, interesting, it seems to still keep all the data locally, but can automatically synchronize them with the central repository.
-
You could have the model used by distributed systems with central storage though and in corporate setup it would make sense.
I have always wanted to look into bazaar, perforce or plastic scm because I believe they claim to be able to do some of this. If only I had the time...
-
which eventually makes svn the most fitting solution for most corporate use-case instead of git.
Besides, there is git-svn if people prefer to use git on their end. Personally, I quite like that setup (svn backend, git frontend).
-
bazaar
Bazaar is git in pale blue, except instead of “content addressing” with hashes it uses generated identifiers (from time, user id and some other stuff). I used to think it was better in the early days until I learned Git a bit more and realized it is actually much more powerful. All the decisions like tracking renames, unique revision identifiers and semi-sequential revision numbers look like good ideas, but they turn out to mainly complicate the model.
perforce or plastic scm
Hm, may be interesting, but these days few companies are going to buy commercial SCM when they can have Git for free.
-
which eventually makes svn the most fitting solution for most corporate use-case instead of git.
Besides, there is git-svn if people prefer to use git on their end. Personally, I quite like that setup (svn backend, git frontend).
We used git-svn at previous workplace. It works reasonably well, as long as absolutely everyone in the project always uses git-svn and never raw SVN (it wasn't true for my project, and it was a constant source of problems.)
-
All the decisions like tracking renames, unique revision identifiers and semi-sequential revision numbers look like good ideas, but they turn out to mainly complicate the model.
AFAIK Git also tracks renames? Also, I never used Bazaar, but I use Mercurial, which also has sequential revision numbers - and yes, they are quite useless. But I like explicit branch management (branch name isn't just an ever-moving tag, but an actual part of commit data - so you always know which branch a particular commit came from even if the branch was already deleted), so for me, Hg > Git.
npm-adduser | npm Docs
