"We spin up 50 node containers" <-- WTF?! or WTF!?
-
@apapadimoulis said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
is nodejs really that deficient of a hosting platform that it can't utilize worker threads to process reqeusts?
Not so much that nodejs doesn't support it. It's more like javascript is designed to be single threaded. If nodejs wants to support threading it'll have to invent its own mechanism, which will most likely break compatibility with browsers and not be popular. A lot of existing JS code assumes there will not be race conditions
@apapadimoulis said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
@Adynathos said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
Is there any point in running more threads than CPU cores?
Yes; you can have threads waiting on things like database calls, etc., that aren't constrained by I/O bottlenecks; regardless, the OS can manage I/O bottlenecks a heck of a lot better than your code (especially if it's JavaScript).
Or, you could not do that and instead have a thread pool to handle these requests using the Reactor pattern? So you don't have to constantly spawn new threads and have them tie up resources while waiting? Or waste time context switching because you have too many threads?
Even your beloved Microsoft stack has implemented this https://msdn.microsoft.com/en-us/library/windows/desktop/aa365198(v=vs.85).aspx
@Adynathos said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
Node is supposedly asynchronous
In the same sense that a Speak & Spell is a word processor, Node is asynchronous.
Single thread != not asynchronous
-
@cark said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
A lot of existing JS code assumes there will not be race conditions
So just Practice and Qualifying?
-
@El_Heffe said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
If someone ever creates Node.NET I will find you and kill you.
Prepare your instruments of death:
-
@Bort said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
Node.net
Node.js
implemented in Javascript
on the .NET runtime
-
@Bort said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
@El_Heffe said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
If someone ever creates Node.NET I will find you and kill you.
Prepare your instruments of death:
-
@error said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
He'll sure be in for a surprise when we show up at his door.
-
BTW .NET core + Kestrel works the same as Node as they both run on libuv.
-
@Adynathos Correct.
It is the same as using a synchronous Ajax request (I know it is an oxymoron) it locks the browser as it stops the event loop, same for alert, confirm etc in a browser.
Python has similar problems with it GIL and you normally get around that by using @asychio (in python 3) or using an event loop (Tornado server for example).
Node is fucking fast if you don't have to wait for anything I/O or heavily computation. I met a guy that was running his business on node.js + express and cassandra. He claimed that it was much better than ASP.NET.
-
@cartman82 If you say have a PHP server doing the same thing. If the request dies, it not a problem. The next request won't get affected, unless you are using an Event Loop PHP framework (they exist).
-
This thread would have been perfect for halloween spooky stories.
-
@cartman82 said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
If your app crashes, it's your own fault. Program better.
Dude, no. Literally all programs crash. Making software tolerant to errors is like the #2 objective of all of software engineering. That's why we have user mode and kernel mode in processors and all that.
-
@lucas1 said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
@cartman82 unless you are using an Event Loop PHP framework (they exist).
-
@anonymous234 said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
Making software tolerant to errors is like the #2 objective of all of software engineering.
I'd put it at about 3 or 4. Getting paid is the #1 objective…
-
@TimeBandit I don't get it.
-
@coderpatsy said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
This thread would have been perfect for halloween spooky stories.
"But according to the flowchart, the NodeJS costume should have been fine!"
-
@dkf said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
@anonymous234 said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
Making software tolerant to errors is like the #2 objective of all of software engineering.
I'd put it at about 3 or 4. Getting paid is the #1 objective…
You just made my little entrepreneur heart happy.
-
@dkf said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
I'd put it at about 3 or 4. Getting paid is the #1 objective…
That depends heavily on a multitude of factors.
-
@masonwheeler said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
That depends heavily on a multitude of factors.
Well, yes. Doesn't change the overall goal of my objective function though.
-
@dkf Yeah, the key word being "my". This is very different from "all software engineering".
-
@Adynathos said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
As far as I understand the Node way of doing things (don't have much experience with it), is that you would register a callback which would be called when the DB response comes back
Which incidentally, is basically what threads do! Only this would be cooperative multitasking instead of preemptive, which means the programmer now has to do the work manually instead of having the system do it automatically.
-
@masonwheeler said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
This is very different from "all software engineering".
OK, but the software engineers who don't put getting paid pretty high up the priority list will get exploited to hell and back by their employers. Using the Kates Principle, getting paid is always a good primary objective.
-
@Adynathos An asynchronous callback is a thread.
Funny story, I had no idea when requests would come in, or the order, however, to handle them it had to be synchronized, because a certain resource couldn't handle being multithreaded very well.
So rather than lock on the 50 times I used the resource in my method, I just used a synchronization thread.
-
@dkf Because open source is totally not a thing that exists at all?
-
@xaade But it is effectively invisible to the programmer in JavaScript.
-
@lucas1 said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
@xaade But it is effectively invisible to the programmer in JavaScript.
Right, but the question was whether having more threads than cores is a benefit. And callbacks demonstrate that benefit.
-
@masonwheeler Obviously, the purpose of "all software engineering" is to get all software engineers paid
-
@Polygeekery said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
You just made my little entrepreneur heart happy.
... and it grew 19.95 sizes that day (plus shipping and handling)...
-
-
@masonwheeler said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
@dkf Because open source is totally not a thing that exists at all?
And yet, only hackers use Linux. To hack. Because they have no money. :Tro
-
@flabdablet said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
@ben_lubar said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
-
@cartman82 said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
@The_Quiet_One said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
Oh, also, NodeJS just crashes if you have an unhandled exception. No, it doesn't just log an error and continues with the next request. It has to be restarted. So, if you have 50 node containers and there's some bug in your software that results in an unhandled exception under some weird circumstance, those containers will start to die one by one.
Newsflash, any code crashes if you have an unhandled exception. The thing that "logs and continues with the next request" just adds a try/catch block for every request. Pick a batteries included framework for node, and it works exactly the same way.
Also, generally, js is high-level enough that exceptions don't happen often. Things normally handled by exceptions in stuff like Java and C# are handled in the error callbacks. If there is an exception, it's more of the
OhShitWhereDidMyFilesystemGoException
or something.
-
@sloosecannon said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
OhShitWhereDidMyFilesystemGoException or something.
I was unaware javascript even dealt with filesystems...
-
@sloosecannon said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
Things normally handled by exceptions in stuff like Java and C# are handled in the error callbacks.
That has nothing to do with the type of errors and everything to do with callback hell throwing away the stack.
-
@sloosecannon said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
Also, generally, js
is high-level enoughignores all manner of reasonable run-time errors so that exceptions don't happen often.FTFY
-
@pydsigner said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
@sloosecannon said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
Also, generally, js
is high-level enoughignores all manner of reasonable run-time errors so that exceptions don't happen often.FTFY
Well yeah that happens too.......
-
@Tsaukpaetra said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
I was unaware javascript even dealt with filesystems...
Does a browser count as an embedded system with no file system?
-
-
@JBert said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
local storage isn't exactly a file system.
it's more...... like a Dictionary<string,string> that you can use, but not forever because if you put too much data in there it suddenly goes "fuck you, no more storage for you!"
-
@accalia said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
@JBert said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
local storage isn't exactly a file system.
it's more...... like a Dictionary<string,string> that you can use, but not forever because if you put too much data in there it suddenly goes "fuck you, no more storage for you!"
And it doesn't even have the courtesy to
warnpay you.
-
@accalia said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
local storage isn't exactly a file system.
it's more...... like a Dictionary<string,string> that you can use, but not forever because if you put too much data in there it suddenly goes "fuck you, no more storage for you!"
Wait, I think you confused local storage with MongoDB
-
@TimeBandit said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
@accalia said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
local storage isn't exactly a file system.
it's more...... like a Dictionary<string,string> that you can use, but not forever because if you put too much data in there it suddenly goes "fuck you, no more storage for you!"
Wait, I think you confused local storage with MongoDB
mongodb is just an extension of localstorage.
-
@accalia said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
like a Dictionary<string,string> that you can use
Why do programmers keep going back to plain old key-value stores? :(
-
@dkf said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
@accalia said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
like a Dictionary<string,string> that you can use
Why do programmers keep going back to plain old key-value stores? :(
because we don't learn the lessons of history very well?
-
@mott555 said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
NodeJS is pretty nifty but it really needs to overcome its threading limitations and provide a better way to share data between instances so it can scale in a non-WTF manner.
No, nodejs should stop being used for web applications because it's a shitty idea. Whoever use it in a web server deserve what happens.
-
@fbmac said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
@mott555 said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
NodeJS is pretty nifty but it really needs to overcome its threading limitations and provide a better way to share data between instances so it can scale in a non-WTF manner.
No, nodejs should stop being used for web applications because it's a shitty idea. Whoever use it in a web server deserve what happens.
I blame "full-stack" developers.
-
Haven't read the whole thing yet, but this seemed appropriate:
-
@dcon said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
Haven't read the whole thing yet, but this seemed appropriate:
huh.... maybe that has something to do with our problems on AWS.
we were using docker there for everything.
@ben_lubar..... are we using docker on the new server or did we do the sane thing and say "screw it"?
-
@apapadimoulis said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
They already demonstrated a lack of competence by choosing NodeJS, so anything beyond that is questionable.
You found something useful about NodeJS. It makes the idiots easier to identify.
-
We're going to use Docker for Windows on Service Fabric, once that goes live.
Probably.
Unless it doesn't work, in which case we can probably escape it.
But we work with a massive legacy product that we want to use as a service, so it kind of makes sense?
-
@Magus said in "We spin up 50 node containers" <-- WTF?! or WTF!?:
Unless it doesn't work, in which case we can probably escape it.
best case it fails fast and hard. worst, and most likely, case it works well enough to get into production and then it starts to fail, but subtly, in interesting ways.