Chrome switches to clang everywhere
-
No more MSVC on Windows. Interesting writeup:
-
Building Chrome locally with Clang is about 15% slower than with MSVC. (We’ve heard that Windows Defender can make Clang builds a lot slower on some machines, so if you’re seeing larger slowdowns, make sure to whitelist Clang in Windows Defender.) However, the way Clang emits debug info is more parallelizable and builds with a distributed build service (e.g. Goma) are hence faster.
If your project is so slow to build that compiling it on a distributed cluster, with all the overhead that that entails, is actually faster than on a single build system, I'm not sure whether that's a bigger indictment of your poor engineering or of the C++ language!
-
@masonwheeler said in Chrome switches to clang everywhere:
Building Chrome locally with Clang is about 15% slower than with MSVC. (We’ve heard that Windows Defender can make Clang builds a lot slower on some machines, so if you’re seeing larger slowdowns, make sure to whitelist Clang in Windows Defender.) However, the way Clang emits debug info is more parallelizable and builds with a distributed build service (e.g. Goma) are hence faster.
If your project is so slow to build that compiling it on a distributed cluster, with all the overhead that that entails, is actually faster than on a single build system, I'm not sure whether that's a bigger indictment of your poor engineering or of the C++ language!
Because you've only worked on small projects?
EDIT: Or languages that JIT compile? Are interpreted?
-
@boomzilla said in Chrome switches to clang everywhere:
@masonwheeler said in Chrome switches to clang everywhere:
Building Chrome locally with Clang is about 15% slower than with MSVC. (We’ve heard that Windows Defender can make Clang builds a lot slower on some machines, so if you’re seeing larger slowdowns, make sure to whitelist Clang in Windows Defender.) However, the way Clang emits debug info is more parallelizable and builds with a distributed build service (e.g. Goma) are hence faster.
If your project is so slow to build that compiling it on a distributed cluster, with all the overhead that that entails, is actually faster than on a single build system, I'm not sure whether that's a bigger indictment of your poor engineering or of the C++ language!
Because you've only worked on small projects?
EDIT: Or languages that JIT compile? Are interpreted?
Yeah, it seems that if you have many hundreds or thousands of individual files to compile (ie more than your processor has available thread space) and they're spread into largely independent modules, that should parallelize beautifully.
-
@boomzilla said in Chrome switches to clang everywhere:
Because you've only worked on small projects?
Does ~7M lines of code count as small?
EDIT: Or languages that JIT compile? Are interpreted?
Nope.
-
@benjamin-hall I just took a look and it appears that the executable (Linux, 64-bit) that I have installed is ~120MB.
-
@masonwheeler said in Chrome switches to clang everywhere:
@boomzilla said in Chrome switches to clang everywhere:
Because you've only worked on small projects?
Does ~7M lines of code count as small?
EDIT: Or languages that JIT compile? Are interpreted?
Nope.
Soooo....you have no excuse then?
-
@benjamin-hall said in Chrome switches to clang everywhere:
Yeah, it seems that if you have many hundreds or thousands of individual files to compile (ie more than your processor has available thread space) and they're spread into largely independent modules, that should parallelize beautifully.
In theory, yes. In practice, it's not easy at all to create a large number of truly independent modules. You end up with a hierarchy of dependencies and interdependencies, which is much less beautiful to parallelize.
-
@boomzilla said in Chrome switches to clang everywhere:
Soooo....you have no excuse then?
What I have is experience in large projects that compile quickly.
-
@boomzilla the key (to my amateur eyes anyway) should be the number of independent units, not the overall size. If you have blocking compiles (must compile A to compile B, etc) then not so much, but if you have dozens/hundreds/thousands of independent units that, in the end, must be linked together you can compile in parallel and do a lot of the sub linking as well, then bring it all together and link the final executable.
-
@masonwheeler said in Chrome switches to clang everywhere:
What I have is experience in large projects that compile quickly.
Chromium is ~18M lines of code. Add a bit more for Chrome
-
@masonwheeler said in Chrome switches to clang everywhere:
@benjamin-hall said in Chrome switches to clang everywhere:
Yeah, it seems that if you have many hundreds or thousands of individual files to compile (ie more than your processor has available thread space) and they're spread into largely independent modules, that should parallelize beautifully.
In theory, yes. In practice, it's not easy at all to create a large number of truly independent modules. You end up with a hierarchy of dependencies and interdependencies, which is much less beautiful to parallelize.
In whose practice? In which language?
-
@boomzilla Today in Implementation Detail News!!!
-
From https://chromium.googlesource.com/chromium/src/+/lkcr/docs/linux_build_instructions.md
It estimates that
Quick and dirty benchmark numbers on a HP Z600 (Intel core i7, 16 cores hyperthreaded, 12 GB RAM)..12m:20s
for building Chromium (once you have all the dependencies). Looking at the repo, they have many thousands of execution units, all of which need to be compiled first. That seems like a distributed build system would help. This is especially true if you want to do a quick-turnaround code-test-code cycle (since you have to build at least a chunk of it each time you run a test).
-
@blakeyrat said in Chrome switches to clang everywhere:
@boomzilla Today in Implementation Detail News!!!
Yes! Pretty interesting, isn't it?
-
@masonwheeler said in Chrome switches to clang everywhere:
Building Chrome locally with Clang is about 15% slower than with MSVC. (We’ve heard that Windows Defender can make Clang builds a lot slower on some machines, so if you’re seeing larger slowdowns, make sure to whitelist Clang in Windows Defender.) However, the way Clang emits debug info is more parallelizable and builds with a distributed build service (e.g. Goma) are hence faster.
If your project is so slow to build that compiling it on a distributed cluster, with all the overhead that that entails, is actually faster than on a single build system, I'm not sure whether that's a bigger indictment of your poor engineering or of the C++ language!
The project I'm working compiles in about 11s in a machine without any anti-virus. It takes about 5min after all the security stuff is installed on it.
-
@sockpuppet7 said in Chrome switches to clang everywhere:
@masonwheeler said in Chrome switches to clang everywhere:
Building Chrome locally with Clang is about 15% slower than with MSVC. (We’ve heard that Windows Defender can make Clang builds a lot slower on some machines, so if you’re seeing larger slowdowns, make sure to whitelist Clang in Windows Defender.) However, the way Clang emits debug info is more parallelizable and builds with a distributed build service (e.g. Goma) are hence faster.
If your project is so slow to build that compiling it on a distributed cluster, with all the overhead that that entails, is actually faster than on a single build system, I'm not sure whether that's a bigger indictment of your poor engineering or of the C++ language!
The project I'm working compiles in about 11s in a machine without any anti-virus. It takes about 5min after all the security stuff is installed on it.
It takes about three hours on our fastest multicore 64 GB machines to compile Unreal Engine and our game (server and client flavors) from a clean state.
-
@tsaukpaetra said in Chrome switches to clang everywhere:
It takes about three hours on our fastest multicore 64 GB machines to compile Unreal Engine and our game (server and client flavors) from a clean state.
-
@timebandit said in Chrome switches to clang everywhere:
@tsaukpaetra said in Chrome switches to clang everywhere:
It takes about three hours on our fastest multicore 64 GB machines to compile Unreal Engine and our game (server and client flavors) from a clean state.
Why do you think I can be so responsive here on the forums most of the time?
-
@tsaukpaetra said in Chrome switches to clang everywhere:
@sockpuppet7 said in Chrome switches to clang everywhere:
@masonwheeler said in Chrome switches to clang everywhere:
Building Chrome locally with Clang is about 15% slower than with MSVC. (We’ve heard that Windows Defender can make Clang builds a lot slower on some machines, so if you’re seeing larger slowdowns, make sure to whitelist Clang in Windows Defender.) However, the way Clang emits debug info is more parallelizable and builds with a distributed build service (e.g. Goma) are hence faster.
If your project is so slow to build that compiling it on a distributed cluster, with all the overhead that that entails, is actually faster than on a single build system, I'm not sure whether that's a bigger indictment of your poor engineering or of the C++ language!
The project I'm working compiles in about 11s in a machine without any anti-virus. It takes about 5min after all the security stuff is installed on it.
It takes about three hours on our fastest multicore 64 GB machines to compile Unreal Engine and our game (server and client flavors) from a clean state.
How many cores and how many files does it compile in parallel? SSDs?
-
@tsaukpaetra said in Chrome switches to clang everywhere:
@sockpuppet7 said in Chrome switches to clang everywhere:
@masonwheeler said in Chrome switches to clang everywhere:
Building Chrome locally with Clang is about 15% slower than with MSVC. (We’ve heard that Windows Defender can make Clang builds a lot slower on some machines, so if you’re seeing larger slowdowns, make sure to whitelist Clang in Windows Defender.) However, the way Clang emits debug info is more parallelizable and builds with a distributed build service (e.g. Goma) are hence faster.
If your project is so slow to build that compiling it on a distributed cluster, with all the overhead that that entails, is actually faster than on a single build system, I'm not sure whether that's a bigger indictment of your poor engineering or of the C++ language!
The project I'm working compiles in about 11s in a machine without any anti-virus. It takes about 5min after all the security stuff is installed on it.
It takes about three hours on our fastest multicore 64 GB machines to compile Unreal Engine and our game (server and client flavors) from a clean state.
We're at 1-2 hours for our stuff.
-
@boomzilla said in Chrome switches to clang everywhere:
@tsaukpaetra said in Chrome switches to clang everywhere:
@sockpuppet7 said in Chrome switches to clang everywhere:
@masonwheeler said in Chrome switches to clang everywhere:
Building Chrome locally with Clang is about 15% slower than with MSVC. (We’ve heard that Windows Defender can make Clang builds a lot slower on some machines, so if you’re seeing larger slowdowns, make sure to whitelist Clang in Windows Defender.) However, the way Clang emits debug info is more parallelizable and builds with a distributed build service (e.g. Goma) are hence faster.
If your project is so slow to build that compiling it on a distributed cluster, with all the overhead that that entails, is actually faster than on a single build system, I'm not sure whether that's a bigger indictment of your poor engineering or of the C++ language!
The project I'm working compiles in about 11s in a machine without any anti-virus. It takes about 5min after all the security stuff is installed on it.
It takes about three hours on our fastest multicore 64 GB machines to compile Unreal Engine and our game (server and client flavors) from a clean state.
How many cores and how many files does it compile in parallel? SSDs?
I think it's a 32 core (hyper thread) from dual Xeon CPUs running on NVMe from Intel. One compiler per virtual CPU.
It's not any
fasterslower than our six-core one that's on an M.2 ssd that's two GHz faster on the clock.
-
@xaade said in Chrome switches to clang everywhere:
@tsaukpaetra said in Chrome switches to clang everywhere:
@sockpuppet7 said in Chrome switches to clang everywhere:
@masonwheeler said in Chrome switches to clang everywhere:
Building Chrome locally with Clang is about 15% slower than with MSVC. (We’ve heard that Windows Defender can make Clang builds a lot slower on some machines, so if you’re seeing larger slowdowns, make sure to whitelist Clang in Windows Defender.) However, the way Clang emits debug info is more parallelizable and builds with a distributed build service (e.g. Goma) are hence faster.
If your project is so slow to build that compiling it on a distributed cluster, with all the overhead that that entails, is actually faster than on a single build system, I'm not sure whether that's a bigger indictment of your poor engineering or of the C++ language!
The project I'm working compiles in about 11s in a machine without any anti-virus. It takes about 5min after all the security stuff is installed on it.
It takes about three hours on our fastest multicore 64 GB machines to compile Unreal Engine and our game (server and client flavors) from a clean state.
We're at 1-2 hours for our stuff.
Once it's done the first round, so long as we don't clear out all the "intermediates" future builds only take about ten minutes. But that first build... Ugh.
-
my assignment takes 2s to compile
-
@bb36e My code takes 0s to compile.
FileUnder: PHP
-
@boomzilla said in Chrome switches to clang everywhere:
Yes! Pretty interesting, isn't it?
I can't think of anything less interesting than what compiler a software project uses!!!
-
@masonwheeler said in Chrome switches to clang everywhere:
Building Chrome locally with Clang is about 15% slower than with MSVC. (We’ve heard that Windows Defender can make Clang builds a lot slower on some machines, so if you’re seeing larger slowdowns, make sure to whitelist Clang in Windows Defender.) However, the way Clang emits debug info is more parallelizable and builds with a distributed build service (e.g. Goma) are hence faster.
If your project is so slow to build that compiling it on a distributed cluster, with all the overhead that that entails, is actually faster than on a single build system, I'm not sure whether that's a bigger indictment of your poor engineering or of the C++ language!
Go ahead, tell us how everybody is
because their software doesn't compile in less than 2 seconds.
-
@topspin said in Chrome switches to clang everywhere:
Go ahead, tell us how everybody is
because their software doesn't compile in less than 2 seconds.Not 2 seconds, but that 7 MLOC project? That built in about 5 minutes.
-
@masonwheeler said in Chrome switches to clang everywhere:
@topspin said in Chrome switches to clang everywhere:
Go ahead, tell us how everybody is
because their software doesn't compile in less than 2 seconds.Not 2 seconds, but that 7 MLOC project? That built in about 5 minutes.
To native code, with optimizations, LTO/PGO and everything?
-
@topspin said in Chrome switches to clang everywhere:
To native code, with optimizations,
Yes
LTO/PGO and everything?
No
-
@tsaukpaetra said in Chrome switches to clang everywhere:
Why do you think I can be so responsive here on the forums most of the time?
QFT!
-
@xaade said in Chrome switches to clang everywhere:
We're at 1-2 hours for our stuff.
When I was at VMware, I think a complete build of everything that went into Workstation was supposed to take around 24hrs. Thank $deity for the modularization.
-
@timebandit said in Chrome switches to clang everywhere:
@bb36e My code takes 0s to compile.
FileUnder: PHP
But years to debug.
-
@blakeyrat said in Chrome switches to clang everywhere:
@boomzilla said in Chrome switches to clang everywhere:
Yes! Pretty interesting, isn't it?
I can't think of anything less interesting than what compiler a software project uses!!!
Then get the fuck out of this thread and go play some video games or whatever it is that you think is interesting.
-
@blakeyrat said in Chrome switches to clang everywhere:
@boomzilla said in Chrome switches to clang everywhere:
Yes! Pretty interesting, isn't it?
I can't think of anything less interesting than what compiler a software project uses!!!
Alternatively, go over the polish dog train thread and watch people argue about logjam.
-
@tsaukpaetra said in Chrome switches to clang everywhere:
It takes about three hours on our fastest multicore 64 GB machines to compile Unreal Engine
That is curious. It took me about an hour to compile UE4 on i7 (or ryzen).
Does your game take long to compile?
-
@adynathos It's VR so it has to compile all three dimensions.
-
@adynathos said in Chrome switches to clang everywhere:
@tsaukpaetra said in Chrome switches to clang everywhere:
It takes about three hours on our fastest multicore 64 GB machines to compile Unreal Engine
That is curious. It took me about an hour to compile UE4 on i7 (or ryzen).
Does your game take long to compile?About 45 minutes clean. And that's triple, because compiling the editor, server, and client versions.
-
@blakeyrat said in Chrome switches to clang everywhere:
@boomzilla said in Chrome switches to clang everywhere:
Yes! Pretty interesting, isn't it?
I can't think of anything less interesting than what compiler a software project uses!!!
What about what bug reporting tool?
-
@xaade said in Chrome switches to clang everywhere:
@blakeyrat said in Chrome switches to clang everywhere:
@boomzilla said in Chrome switches to clang everywhere:
Yes! Pretty interesting, isn't it?
I can't think of anything less interesting than what compiler a software project uses!!!
What about what bug reporting tool?
Oh, great, now he's going to get all uppity and brag about his law that every thread eventually becomes a thread about Discourse.
-
@boomzilla I mean, only because you brought it up.
-
@pie_flavor said in Chrome switches to clang everywhere:
@boomzilla I mean, only because you brought it up.
Not before xaade did.
-
"Our browser is so shitty and bloated that the only way to fix it is to switch programming languages and maybe create our own programming language! That is a good decision and not piling epic amounts of shit ontop of an already epic shit-pile"
Huh, well, whaddya know. For once, Chrome is copying Mozilla.
Filed under: rusty dick
-
@lorne-kates said in Chrome switches to clang everywhere:
Huh, well, whaddya know. For once, Chrome is copying Mozilla.
Uh, no. They're still using C++. Just using a single compiler for all platforms.
-
@lorne-kates Hey, Rust was a good decision by Mozilla. If it manages to take off it will probably be more important than Firefox.
-
If they write Chrome in Go then I'll laugh.
-
@anonymous234 said in Chrome switches to clang everywhere:
@lorne-kates Hey, Rust was a good decision by Mozilla. If it manages to take off it will probably be more important than Firefox.
Let's ask
The language grew out of a personal project started in 2006 by Mozilla employee Graydon Hoare, ... Mozilla began sponsoring the project in 2009 ... The first numbered pre-alpha release of the Rust compiler occurred in January 2012.[33] Rust 1.0, the first stable release, was released on May 15, 2015
9 years from a self-wank to a "whatever Mozilla calls stable".
Yeah, taking the world by storm.
Hmmm... I wonder what happens if I google "Breaking Changes Rust".
First hit: https://killercup.github.io/bitrust/
First result: dee8a71c: fix #[derive] implementation for repr(packed) structs - Ariel Ben-Yehuda, 2017-11-26 09:12:43 -0500
So 11 years after conception and 2 years after "Stable", they're still introducing breaking changes. To what they want to be a widely accepted "better than others" language.
-
@lorne-kates said in Chrome switches to clang everywhere:
@anonymous234 said in Chrome switches to clang everywhere:
@lorne-kates Hey, Rust was a good decision by Mozilla. If it manages to take off it will probably be more important than Firefox.
Let's ask
The language grew out of a personal project started in 2006 by Mozilla employee Graydon Hoare, ... Mozilla began sponsoring the project in 2009 ... The first numbered pre-alpha release of the Rust compiler occurred in January 2012.[33] Rust 1.0, the first stable release, was released on May 15, 2015
9 years from a self-wank to a "whatever Mozilla calls stable".
Yeah, taking the world by storm.
Hmmm... I wonder what happens if I google "Breaking Changes Rust".
First hit: https://killercup.github.io/bitrust/
First result: dee8a71c: fix #[derive] implementation for repr(packed) structs - Ariel Ben-Yehuda, 2017-11-26 09:12:43 -0500
So 11 years after conception and 2 years after "Stable", they're still introducing breaking changes. To what they want to be a widely accepted "better than others" language.
It's amazing that that's the metric you're measuring it by. If you actually knew what those fixes were, you'd realize that they're only breaking code that does things seriously wrong. Not to mention that nobody is forcing you to use the latest compiler; you can build using whatever version you want and keep your builds deterministic. But if you resolve to literally never break anything ever, then your language becomes shit. Look at Java.
-
@pie_flavor said in Chrome switches to clang everywhere:
But if you resolve to literally never break anything ever, then your language becomes shit. Look at Java.
Someone actually bought the bridge that Oracle was selling?
I mean, fuck, even Oracle's own products randomly break between Java versions. I'm looking at you, Oracle Application Server.
-
@lorne-kates said in Chrome switches to clang everywhere:
So 11 years after conception and 2 years after "Stable", they're still introducing breaking changes.
Oh man, just wait until you hear about C++11 (
auto), C++14 (std::gets), C++17 (std::mem_fun), C++20... same explanation as @pie_flavor
