I'm feeling happy with this, and it isn't something I wrote by myself, but we've just got an “accurate” power measurement instrumentation system written at work. It's pretty neat, and generates a nice report at the end of the simulation describing how many Joules of energy were used at each stage of the processing. Right now, we're still in the process of validating it (because we were initially checking with a trivial example simulation so that tests would be very quick) but it's looking good. It'll replace our previous system for doing this, which was a board with an Arduino bolted on the side to do power measurement, which would be fine except that it doesn't sample nearly often enough to catch meaningful detail (also there's only one such board and it appears to have some minor hardware faults).

I wrote the part that actually measures the power usage of the CPUs, taking into account all the tricky bits like possible systematic sampling errors and running at a high enough frequency to be actually able to detect the detailed behaviour of our simulation engine. Which sounds trivial, and sort of is, but tends to cause problems if you don't plug in code to ensure that you do it right.

We're planning to totally brag about this capability in our future papers.