Microsoft knows themuir lthreadtiing
-
Output from nmake while trying to build QT:
NNMMNAAMKKAEEK :E: f:fa atftaaaltl a eler rrerororrr o UrU1 10U0515808:5: 8 t:te ertrmemirinmnaiatnteaedtd e bdby y b uyus seuersr e SrSt toSoptp.o. p .
EDIT: Yes it says (or attempts to say) "terminated by user" - it was outputting much more error messages than there was room for in my terminals buffer so I ctrl+c'ed it.
-
Wow. And I thought BenL was merely kidding when he made that "use multithreading. Now you problems two have" joke.
It's very appropriate presentation for a fatal error, though. Very "medium is the message"-y.
-
@poizan42 said:
Output from nmake while trying to build QT:
NNMMNAAMKKAEEK :E: f:fa atftaaaltl a eler rrerororrr o UrU1 10U0515808:5: 8 t:te ertrmemirinmnaiatnteaedtd e bdby y b uyus seuersr e SrSt toSoptp.o. p .
Are you sure that's a multithreading problem? That can happen if STDOUT and STDERR are directed to the same place.
-
@RaceProUK said:
It's three nmake instances running in parallel I think. That is at least one thread per instance - so definitely multithreading. The bigger question is why their output ended up being interleaved at character boundaries.@poizan42 said:
Output from nmake while trying to build QT:
NNMMNAAMKKAEEK :E: f:fa atftaaaltl a eler rrerororrr o UrU1 10U0515808:5: 8 t:te ertrmemirinmnaiatnteaedtd e bdby y b uyus seuersr e SrSt toSoptp.o. p .
Are you sure that's a multithreading problem? That can happen if STDOUT and STDERR are directed to the same place.
-
@RaceProUK said:
@poizan42 said:
Output from nmake while trying to build QT:
NNMMNAAMKKAEEK :E: f:fa atftaaaltl a eler rrerororrr o UrU1 10U0515808:5: 8 t:te ertrmemirinmnaiatnteaedtd e bdby y b uyus seuersr e SrSt toSoptp.o. p .
Are you sure that's a multithreading problem? That can happen if STDOUT and STDERR are directed to the same place.
Well, yeah, but that in itself is another multithreading problem.
-
@poizan42 said:
@RaceProUK said:
It's three nmake instances running in parallel I think. That is at least one thread per instance - so definitely multithreading. The bigger question is why their output ended up being interleaved at character boundaries.@poizan42 said:
Output from nmake while trying to build QT:
NNMMNAAMKKAEEK :E: f:fa atftaaaltl a eler rrerororrr o UrU1 10U0515808:5: 8 t:te ertrmemirinmnaiatnteaedtd e bdby y b uyus seuersr e SrSt toSoptp.o. p .
Are you sure that's a multithreading problem? That can happen if STDOUT and STDERR are directed to the same place.
It is multithreading of sorts, but the fault lies with running three instances in parallel, all pointing to a non-threadsafe console output. Not sure what you could do to prevent that happening, not without either losing parallelism or replacing the console.
-
@RaceProUK said:
@poizan42 said:
@RaceProUK said:
@poizan42 said:
Output from nmake while trying to build QT:
Are you sure that's a multithreading problem? That can happen if STDOUT and STDERR are directed to the same place.
It's three nmake instances running in parallel I think. That is at least one thread per instance - so definitely multithreading. The bigger question is why their output ended up being interleaved at character boundaries.It is multithreading of sorts, but the fault lies with running three instances in parallel, all pointing to a non-threadsafe console output. Not sure what you could do to prevent that happening, not without either losing parallelism or replacing the console.
Sane operating systems gives a guarantee of atomicity when writing buffers up to a certain size to pipes. Not surprisingly Windows isn't one of those.
-
@poizan42 said:
@RaceProUK said:
Windows totally does do that, if you use PIPE_TYPE_MESSAGE, but no pipes are involved in this situation.
Sane operating systems gives a guarantee of atomicity when writing buffers up to a certain size to pipes. Not surprisingly Windows isn't one of those.@poizan42 said:
@RaceProUK said:
It's three nmake instances running in parallel I think. That is at least one thread per instance - so definitely multithreading. The bigger question is why their output ended up being interleaved at character boundaries.@poizan42 said:
Output from nmake while trying to build QT:
Are you sure that's a multithreading problem? That can happen if STDOUT and STDERR are directed to the same place.
It is multithreading of sorts, but the fault lies with running three instances in parallel, all pointing to a non-threadsafe console output. Not sure what you could do to prevent that happening, not without either losing parallelism or replacing the console.
-
@DaveK said:
@poizan42 said:
@RaceProUK said:
Windows totally does do that, if you use PIPE_TYPE_MESSAGE, but no pipes are involved in this situation.@poizan42 said:
@RaceProUK said:
@poizan42 said:
Output from nmake while trying to build QT:
Are you sure that's a multithreading problem? That can happen if STDOUT and STDERR are directed to the same place.
It's three nmake instances running in parallel I think. That is at least one thread per instance - so definitely multithreading. The bigger question is why their output ended up being interleaved at character boundaries.It is multithreading of sorts, but the fault lies with running three instances in parallel, all pointing to a non-threadsafe console output. Not sure what you could do to prevent that happening, not without either losing parallelism or replacing the console.
Sane operating systems gives a guarantee of atomicity when writing buffers up to a certain size to pipes. Not surprisingly Windows isn't one of those.Stdout and stderr are pipes in both the posix sense and the Windows API sense in this case, so pipes are very much involved. When I was referering to pipes before I meant byte oriented FIFOs in the posix sense. Message pipes in Windows are closer to sockets in the posix sense
-
@poizan42 said:
@DaveK said:
No, they aren't. They are console handles, which are a very different beast.@poizan42 said:
@RaceProUK said:
Windows totally does do that, if you use PIPE_TYPE_MESSAGE, but no pipes are involved in this situation.
Sane operating systems gives a guarantee of atomicity when writing buffers up to a certain size to pipes. Not surprisingly Windows isn't one of those.@poizan42 said:
@RaceProUK said:
It's three nmake instances running in parallel I think. That is at least one thread per instance - so definitely multithreading. The bigger question is why their output ended up being interleaved at character boundaries.@poizan42 said:
Output from nmake while trying to build QT:
Are you sure that's a multithreading problem? That can happen if STDOUT and STDERR are directed to the same place.
It is multithreading of sorts, but the fault lies with running three instances in parallel, all pointing to a non-threadsafe console output. Not sure what you could do to prevent that happening, not without either losing parallelism or replacing the console.
Stdout and stderr are pipes in both the posix sense and the Windows API sense in this case, so pipes are very much involved.
-
Guys you don't have to quote the full post every fucking time.
Next time when having a catty pedantic debate over whether consoles count as pipes or not, please be less spammy. And also don't have catty pedantic debates over whether consoles count as pipes or not because who fucking cares.
-
@blakeyrat said:
@DaveK said:
@poizan42 said:
@DaveK said:
No, they aren't. They are console handles, which are a very different beast.@poizan42 said:
@RaceProUK said:
Windows totally does do that, if you use PIPE_TYPE_MESSAGE, but no pipes are involved in this situation.@poizan42 said:
@RaceProUK said:
@poizan42 said:
Output from nmake while trying to build QT:
Are you sure that's a multithreading problem? That can happen if STDOUT and STDERR are directed to the same place.
It's three nmake instances running in parallel I think. That is at least one thread per instance - so definitely multithreading. The bigger question is why their output ended up being interleaved at character boundaries.It is multithreading of sorts, but the fault lies with running three instances in parallel, all pointing to a non-threadsafe console output. Not sure what you could do to prevent that happening, not without either losing parallelism or replacing the console.
Sane operating systems gives a guarantee of atomicity when writing buffers up to a certain size to pipes. Not surprisingly Windows isn't one of those.Stdout and stderr are pipes in both the posix sense and the Windows API sense in this case, so pipes are very much involved.
Guys you don't have to quote the full post every fucking time.Next time when having a catty pedantic debate over whether consoles count as pipes or not, please be less spammy. And also don't have catty pedantic debates over whether consoles count as pipes or not because who fucking cares.
Ahahahaha do you see what I did there?
-
@DaveK said:
No, they aren't. They are console handles, which are a very different beast.
Okey so maybe they aren't pipes in the strict Windows API sense. They are still byte oriented FIFOs (or maybe word (as in 16-bit ints) oriented FIFOs - what happens if you write an odd number of bytes to a wide character console handle anyways?). All I wanted was just some sort of atomicity to trivially sized writes so I could output one continous piece of text from one process without having it mixed up with text from the other processes.
Another thing is that my system has a single dual core cpu. How did it manage to mix up three simultanously outputted strings that you would think would take much less than one quantum length (about 50-200 ms) to output?
-
@poizan42 said:
@DaveK said:
Windows, like all sane OSs, does provide atomicity to trivially sized writes, at least as far as I could see by running this testcase and failing to see any intermingled output:No, they aren't. They are console handles, which are a very different beast.
Okey so maybe they aren't pipes in the strict Windows API sense. They are still byte oriented FIFOs (or maybe word (as in 16-bit ints) oriented FIFOs - what happens if you write an odd number of bytes to a wide character console handle anyways?). All I wanted was just some sort of atomicity to trivially sized writes so I could output one continous piece of text from one process without having it mixed up with text from the other processes.
#include <stdio.h>
#include <windows.h>
#define NUM_THREADS 33
char outstring[]]="1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyzIs this an atomic write?";
DWORD WINAPI output_thread (LPVOID param)
{
HANDLE outhan = (HANDLE) param;
DWORD actual;
while (1)
{
(void) WriteFile (outhan, outstring, sizeof(outstring)-1, &actual, NULL);
}
}
int main(int argc, char* argv[])
{
printf("Hello World!\n");
HANDLE outhan = GetStdHandle(STD_OUTPUT_HANDLE);
DWORD threadid[NUM_THREADS];
for (int n = 0; n < NUM_THREADS; n++)
CreateThread (NULL, 0, output_thread, outhan, 0, &threadid[n]);
while (1) { };
return 0;
}
It must be the MSVCRT that gets it wrong.
-
Oh, and to answer your question:
@poizan42 said:
They are still byte oriented FIFOs (or maybe word (as in 16-bit ints) oriented FIFOs - what happens if you write an odd number of bytes to a wide character console handle anyways?).
That's not even possible, as you'd know if you weren't posting uninformed speculation on a subject you have no knowledge of.
-
@DaveK said:
Filed under: blakey is never happy so who fucking cares?
It's just so trivial. I don't see how you guys can give a shit. The guy at Microsoft who WROTE this code probably only barely gave a shit.
-
@DaveK said:
That's not even possible, as you'd know if you weren't posting uninformed speculation on a subject you have no knowledge of.
Hey, but I post uninformed speculation on subjects I have no knowledge of!
-
@blakeyrat said:
@DaveK said:
Filed under: blakey is never happy so who fucking cares?
It's just so trivial. I don't see how you guys can give a shit. The guy at Microsoft who WROTE this code probably only barely gave a shit.
Interesting gambit. Perhaps I'll start doing this on the video game threads.
Oops. Almost forgot.
@DaveK said:
@poizan42 said:
@RaceProUK said:
Windows totally does do that, if you use PIPE_TYPE_MESSAGE, but no pipes are involved in this situation.@poizan42 said:
@RaceProUK said:
@poizan42 said:
Output from nmake while trying to build QT:
Are you sure that's a multithreading problem? That can happen if STDOUT and STDERR are directed to the same place.
It's three nmake instances running in parallel I think. That is at least one thread per instance - so definitely multithreading. The bigger question is why their output ended up being interleaved at character boundaries.It is multithreading of sorts, but the fault lies with running three instances in parallel, all pointing to a non-threadsafe console output. Not sure what you could do to prevent that happening, not without either losing parallelism or replacing the console.
Sane operating systems gives a guarantee of atomicity when writing buffers up to a certain size to pipes. Not surprisingly Windows isn't one of those.
-
@blakeyrat said:
@DaveK said:
Oh, you have no... interest in the world.Filed under: blakey is never happy so who fucking cares?
It's just so trivial. I don't see how you guys can give a shit. The guy at Microsoft who WROTE this code probably only barely gave a shit.
-
@DaveK said:
Oh, and to answer your question:
@poizan42 said:
They are still byte oriented FIFOs (or maybe word (as in 16-bit ints) oriented FIFOs - what happens if you write an odd number of bytes to a wide character console handle anyways?).
That's not even possible, as you'd know if you weren't posting uninformed speculation on a subject you have no knowledge of.I thank you for your insightful and educating comment sir. To answer my own question:
SetConsoleOutputCP refuses to set the codepage to 1200/1201 (UTF-16LE/UTF-16BE), so we can't WriteFile UTF-16 strings to the console.
WriteConsoleOutput wants the number of characters and not bytes to write, so you can't write incomplete wide chars (more precisely you can't write incomplete UTF-16 code units - you can still write half of a surrogate pair).
There are still other multibyte encodings left. Writing incomplete or wrong data with WriteFile results in replacement characters in their stead. Example:#define WIN32_LEAN_AND_MEAN #include <windows.h> #include <stdio.h>
int wmain(int argc, wchar_t *argv[], wchar_t *envp[])
{
HANDLE out = GetStdHandle(STD_OUTPUT_HANDLE);
LPCWSTR utf16le = L"æøå";
char utf16be[] = { 0xE6, 0, 0xF8, 0, 0xE5, 0 };
char utf8[] = { 0xC3, 0xA6, 0xC3, 0xB8, 0xC3, 0xA5 };
UINT cp = GetConsoleOutputCP();
printf("UTF-16LE:\n");
if (!SetConsoleOutputCP(1200)) //UTF-16LE
printf("Set UTF-16LE failed.");
else
WriteFile(out, utf16le, 6, NULL, NULL);
SetConsoleOutputCP(cp);
printf("\nUTF-16BE:\n");
if (!SetConsoleOutputCP(1201)) //UTF-16BE
printf("Set UTF-16BE failed.");
else
WriteFile(out, utf16be, 6, NULL, NULL);
SetConsoleOutputCP(cp);
printf("\nUTF-8:\n");
if (!SetConsoleOutputCP(65001)) //UTF-8
printf("Set UTF-8 failed.");
else
WriteFile(out, utf8, 6, NULL, NULL);
SetConsoleOutputCP(cp);
printf("\nUTF-8 partial string:\n");
if (!SetConsoleOutputCP(65001)) //UTF-8
printf("Set UTF-8 failed.");
else
WriteFile(out, utf8, 5, NULL, NULL);
SetConsoleOutputCP(cp);
printf("\nfin\n");
return 0;
}
Output:UTF-16LE: Set UTF-16LE failed. UTF-16BE: Set UTF-16BE failed. UTF-8: æøå UTF-8 partial string: æø� fin
-
@eViLegion said:
@blakeyrat said:
@DaveK said:
@poizan42 said:
@DaveK said:
No, they aren't. They are console handles, which are a very different beast.@poizan42 said:
@RaceProUK said:
Windows totally does do that, if you use PIPE_TYPE_MESSAGE, but no pipes are involved in this situation.
Sane operating systems gives a guarantee of atomicity when writing buffers up to a certain size to pipes. Not surprisingly Windows isn't one of those.@poizan42 said:
@RaceProUK said:
It's three nmake instances running in parallel I think. That is at least one thread per instance - so definitely multithreading. The bigger question is why their output ended up being interleaved at character boundaries.@poizan42 said:
Output from nmake while trying to build QT:
NNMMNAAMKKAEEK :E: f:fa atftaaaltl a eler rrerororrr o UrU1 10U0515808:5: 8 t:te ertrmemirinmnaiatnteaedtd e bdby y b uyus seuersr
e
SrSt
toSoptp.o.
p
.Are you sure that's a multithreading problem? That can happen if STDOUT and STDERR are directed to the same place.
It is multithreading of sorts, but the fault lies with running three instances in parallel, all pointing to a non-threadsafe console output. Not sure what you could do to prevent that happening, not without either losing parallelism or replacing the console.
Stdout and stderr are pipes in both the posix sense and the Windows API sense in this case, so pipes are very much involved.
Guys you don't have to quote the full post every fucking time.Next time when having a catty pedantic debate over whether consoles count as pipes or not, please be less spammy. And also don't have catty pedantic debates over whether consoles count as pipes or not because who fucking cares.
Ahahahaha do you see what I did there?
For what it's worth, I see this a lot on Linux as well.
-
-
@Ronald said:
@poizan42 said:
QT
Everybody missed it because of the pipes stuff but that's TRWTF
teehee You're the real QT!
-
-
@eViLegion said:
@blakeyrat said:
@DaveK said:
@poizan42 said:
@DaveK said:
No, they aren't. They are console handles, which are a very different beast.@poizan42 said:
@RaceProUK said:
Windows totally does do that, if you use PIPE_TYPE_MESSAGE, but no pipes are involved in this situation.
Sane operating systems gives a guarantee of atomicity when writing buffers up to a certain size to pipes. Not surprisingly Windows isn't one of those.@poizan42 said:
@RaceProUK said:
It's three nmake instances running in parallel I think. That is at least one thread per instance - so definitely multithreading. The bigger question is why their output ended up being interleaved at character boundaries.@poizan42 said:
Output from nmake while trying to build QT:
Are you sure that's a multithreading problem? That can happen if STDOUT and STDERR are directed to the same place.
It is multithreading of sorts, but the fault lies with running three instances in parallel, all pointing to a non-threadsafe console output. Not sure what you could do to prevent that happening, not without either losing parallelism or replacing the console.
Stdout and stderr are pipes in both the posix sense and the Windows API sense in this case, so pipes are very much involved.
Guys you don't have to quote the full post every fucking time.Next time when having a catty pedantic debate over whether consoles count as pipes or not, please be less spammy. And also don't have catty pedantic debates over whether consoles count as pipes or not because who fucking cares.
Ahahahaha do you see what I did there?
No, I don't <wicked grin>
-
@TheCPUWizard said:
@eViLegion said:
@blakeyrat said:
@DaveK said:
@poizan42 said:
@DaveK said:
No, they aren't. They are console handles, which are a very different beast.@poizan42 said:
@RaceProUK said:
Windows totally does do that, if you use PIPE_TYPE_MESSAGE, but no pipes are involved in this situation.
Sane operating systems gives a guarantee of atomicity when writing buffers up to a certain size to pipes. Not surprisingly Windows isn't one of those.@poizan42 said:
@RaceProUK said:
It's three nmake instances running in parallel I think. That is at least one thread per instance - so definitely multithreading. The bigger question is why their output ended up being interleaved at character boundaries.@poizan42 said:
Output from nmake while trying to build QT:
Are you sure that's a multithreading problem? That can happen if STDOUT and STDERR are directed to the same place.
It is multithreading of sorts, but the fault lies with running three instances in parallel, all pointing to a non-threadsafe console output. Not sure what you could do to prevent that happening, not without either losing parallelism or replacing the console.
Stdout and stderr are pipes in both the posix sense and the Windows API sense in this case, so pipes are very much involved.
Guys you don't have to quote the full post every fucking time.Next time when having a catty pedantic debate over whether consoles count as pipes or not, please be less spammy. And also don't have catty pedantic debates over whether consoles count as pipes or not because who fucking cares.
Ahahahaha do you see what I did there?
No, I don't <wicked grin>
I think he's making reference to the "I see what you did there, meme". I could be wrong.
-
@stinerman said:
What's round on the ends and tall in the middle?
Captain Future's spaceship (Comet)?
-
@MiffTheFox said:
@poizan42 said:
#define WIN32_LEAN_AND_MEAN
You didn't even read the linked article, did you? It may not have much real effect nowadays unless you use socket calls in which case it is (kinda) required (unless you want to use olde schoole winsock 1). Also it never hurts to make the compiler's job a bit easier so you might as well just use it when you don't have a need for anything in the excluded headers.
-
@poizan42 said:
[quote user="MiffTheFox"][quote user="poizan42"]#define WIN32_LEAN_AND_MEAN
That doesn't even mean anything anymore.
[/quote]You didn't even read the linked article, did you? It may not have much real effect nowadays unless you use socket calls in which case it is (kinda) required (unless you want to use olde schoole winsock 1). Also it never hurts to make the compiler's job a bit easier so you might as well just use it when you don't have a need for anything in the excluded headers.[/quote]
TRWTF is considering the comments part of the article. I read the article (which said that the lean and mean header was just ignored), not the comments.
No wait, TRWTF is writing native code for Windows. At least DaveK's code didn't require any __underline_prefixed_symbols even though it did have the ARBITRARYUPPERCASE.
-
@MiffTheFox said:
TRWTF is considering the comments part of the article. I read the article (which said that the lean and mean header was just ignored), not the comments.
What, why are you bringing up the comments?
You clearly didn't read the article because the article itself said that it's relatively useless because of how fast computers are now, not that it doesn't do anything.
-
@MiffTheFox said:
Since I was trying to show the behaviour of the underlying OS, as distinct from the C runtime library, it was pretty essential that I did it that way.No wait, TRWTF is writing native code for Windows. At least DaveK's code didn't require any __underline_prefixed_symbols even though it did have the ARBITRARYUPPERCASE.
(The _underline_prefixed_symbols are part of the CRT, btw.)
-
-
@dhromed said:
@DaveK said:
(The _underline_prefixed_symbols are part of the CRT, btw.)
Cathode Ray Tube?
C Run Time. Please don't ask what C means.
-
@boomzilla said:
@dhromed said:
The C is short for CRT.@DaveK said:
(The _underline_prefixed_symbols are part of the CRT, btw.)
Cathode Ray Tube?
C Run Time. Please don't ask what C means.
-
@DaveK said:
@boomzilla said:
@dhromed said:
The C is short for CRT.@DaveK said:
(The _underline_prefixed_symbols are part of the CRT, btw.)
Cathode Ray Tube?
C Run Time. Please don't ask what C means.
I thought it was called
libc
-
@boomzilla said:
@dhromed said:
Top contenders: Carbon, Celsius, Roman numeral for one hundred.@DaveK said:
(The _underline_prefixed_symbols are part of the CRT, btw.)
Cathode Ray Tube?
C Run Time. Please don't ask what C means.
Second-tier contenders: ascorbic acid, mid-range letter grade on a school test, common dry-cell battery size.
Improbable outliers: speed of light, white-key piano major scale.
