Interesting points, eagle. I tend to agree with your philosophy on optimization, i.e., readability first, and performance second, in fact I generally take this beyond the realm string concatenation (that is of course, unless I run into performance problems).
I have a quick question about your code example though. Assuming the String "a" is read later on in the code, would Sun's optimizer simply just omit the first two lines of concatenation? They have no effect on the program, and it seems like this is something a reasonbly well written optimizer should be able to determine (I understand what you're trying to demonstrate, just asking 'cos you seem to know a bit about optimizers).
Also, in your third concatenation example, may I suggest:
a = b + (what ? c : "") + d;
This is, IMO, the most readable way to accomplish what we're trying to accomplish (it clearly shows that regardless of what "what" is, b and d are going to be the front and back of the string, and c may be added, depending on the value of "what"). Also, it's only marginally less optimal than your example (i.e., when "what" is false, we're still stuck with two appends, rather than just one).