We need to monitor the heap size
-
After our peer team rehacked their system to save less data, they started complaining that their system had experienced a significant performance issue. They were running JProbe to try and analyze the performance problem, but just couldn't figure it out. Since I'm the resident threading expert (at least around here), they asked for my help. I found this being called after every single logical transaction:
public void saveHeap() {
// grab max heap limit, current heap size and free bytes from Runtime
// store info to a logging table in the db, for subsequent analysis
}When I pointed it out, they insisted that it wasn't the problem as it was just being run once a transaction, and in straight-line code.
I pointed out that this straight-line code was being run in a thread, which was one of 100 threads, all doing the same thing in parallel.
I looked in the log table and there were tens of millions of rows.
...But we need to monitor heap size!
-
Why is there a table in the DB that allows them to do this? That's just asking for trouble.
-
@Evilweasil said:
They wanted to be able to store it for longer than the log rotation period, so just logging the data wasn't sufficient.Why is there a table in the DB that allows them to do this? That's just asking for trouble.
I don't mind them storing it in the db. I don't even mind them taking a snapshot after every transaction, but every thread is reporting the same information - they could cut out 99.999% of the overhead by just doing it once a minute on a timer in its own thread. These guys just don't comprehend parallel processing.
-
If they fail to comprehend parallel processing, perhaps they should only be entrusted with one thread.
Or maybe a mop. You can tell them I said that, too.
-
@Evilweasil said:
Nah, a mop has too many threads. they'd get lost trying to put it in the bucket.If they fail to comprehend parallel processing, perhaps they should only be entrusted with one thread.
Or maybe a mop. You can tell them I said that, too.
-
-
Snoofle, I've got a question for you. How do you avoid workplace violence?
-
@locallunatic said:
How do you know he does? Maybe he's just got really good at hiding the evidence of said violence...Snoofle, I've got a question for you. How do you avoid workplace violence?
-
@locallunatic said:
I stopped taking other peoples' stupidity personally a long time ago. For me, it's the satisfaction of fixing the mess; openly mocking the idiot is a bonus.Snoofle, I've got a question for you. How do you avoid workplace violence?
-
What I don't understand is why Alex Papadimoulis doesn't open a Snoffle category and lets you post your articles there on the front page.
-
@snoofle said:
These guys just don't comprehend parallel processing.
It's kind of like recursion. To understand recursion you have to, uh, well, there's that calling stuff and, you know, some complex stuff, and uh, you can't do that in this language so it doesn't matter so just forget about it.
-
@TheRider said:
What I don't understand is why Alex Papadimoulis doesn't open a Snoffle category and lets you post your articles there on the front page.
Because the front page is like Oracle. Whenever something new is assimilated by the moloch, the thing withers, loses colour, and becomes incoherent zombie-like shadow of its past self.
-
@SteamBoat said:
It's kind of like recursion. To understand recursion you have to, uh, well, there's that calling stuff and, you know
You can learn more about recursion here.
-
@bannedfromcoding said:
@TheRider said:
Are you saying we should bring back "Mandatory Fun Day"?What I don't understand is why Alex Papadimoulis doesn't open a Snoffle category and lets you post your articles there on the front page.
Because the front page is like Oracle. Whenever something new is assimilated by the moloch, the thing withers, loses colour, and becomes incoherent zombie-like shadow of its past self.I'm starting a petition then.
-
Snoofle could be the next "House MD", just replace the medical nonsense words with software nonsense words.
"It's not Lupus" -> "It's not a Rootkit"
-
Oh my God!
What are the lines of thinking to reach this conclusion? Didn't ANY developer from that team thought that it was a VERY VERY VERY stupid idea? Do they know nothing about async queues, logging rotation, threading, database performance?
I've seen WTFs here which are plain stupid, but this has reached a point of epic fail, because, it doesn't even work! Well, the heap is shared by all thread, so having 100's of thread dumping its size is totally irrelevant.
As an added bonus, @snoofle, did someone ever looked at those metrics? Bet no.
For future reference on a way of doing this EASILY, there's this thing called JMX, read about it, learn it and use it.
-
@snoofle said:
These guys just don't comprehend parallel processing.
Parallel processing is a form of schizophrenia. Imagine having 100 little gremlins living inside your brain. Could become quite exciting!
- I'm schizophrenic, and so am I. Me too!
-
@bgodot said:
Snoofle could be the next "House MD", just replace the medical nonsense words with software nonsense words.
"It's not Lupus" -> "It's not a Rootkit"
Maybe it's a segfault? "It's never a segfault!"
-
@Cassidy said:
That's not really it. You can actually learn more about recursion here.@SteamBoat said:
It's kind of like recursion. To understand recursion you have to, uh, well, there's that calling stuff and, you know
You can learn more about recursion here.
-
@DaveK said:
@Cassidy said:
@SteamBoat said:
That's not really it. You can actually learn more about recursion here.It's kind of like recursion. To understand recursion you have to, uh, well, there's that calling stuff and, you know
You can learn more about recursion here.Here's a puzzler: which joke is least funny? This "link to the same thread and call it a manual on recursion" joke? (Bonus points for making the EXACT SAME "JOKE" twice in the same thread.) Or the "replace words in Shakespeare with computer terms" joke?
The answer is if you found either funny, seppuku immediately.
Protip: seppuku is inherently funny.
-
@blakeyrat said:
@DaveK said:
I didn't find either funny, but in general I'm smart enough to not read, or just skim, threads that I don't enjoy reading.@Cassidy said:
@SteamBoat said:
That's not really it. You can actually learn more about recursion here.It's kind of like recursion. To understand recursion you have to, uh, well, there's that calling stuff and, you know
You can learn more about recursion here.Here's a puzzler: which joke is least funny? This "link to the same thread and call it a manual on recursion" joke? (Bonus points for making the EXACT SAME "JOKE" twice in the same thread.) Or the "replace words in Shakespeare with computer terms" joke?
The answer is if you found either funny, seppuku immediately.
Protip: seppuku is inherently funny.
-
@Sutherlands said:
I didn't find either funny, but in general I'm smart enough to not read, or just skim, threads that I don't enjoy reading.
You mean we're not supposed to continually gripe, moan and whine at everything we don't find funny on here? Oh dear, someone's doing it wrong then...
-
@Sutherlands said:
@blakeyrat said:
You're probably also smart enough to learn more about operating systems and programs you're expected to work with to find information that will make your life easier.Here's a puzzler: which joke is least funny? This "link to the same thread and call it a manual on recursion" joke? (Bonus points for making the EXACT SAME "JOKE" twice in the same thread.) Or the "replace words in Shakespeare with computer terms" joke?
I didn't find either funny, but in general I'm smart enough to not read, or just skim, threads that I don't enjoy reading.The answer is if you found either funny, seppuku immediately.
Protip: seppuku is inherently funny.
-
@toon said:
@bgodot said:
Snoofle could be the next "House MD", just replace the medical nonsense words with software nonsense words.
"It's not Lupus" -> "It's not a Rootkit"
Maybe it's a segfault? "It's never a segfault!""Except when it's a segfault".
-
@blakeyrat said:
Here's a puzzler: which joke is least funny? This "link to the same thread and call it a manual on recursion" joke? (Bonus points for making the EXACT SAME "JOKE" twice in the same thread.) Or the "replace words in Shakespeare with computer terms" joke?
The answer is if you found either funny, seppuku immediately.
Protip: seppuku is inherently funny.
The recursion joke might have been funny if Google hadn't beat them to the punch.
Some of the Shakespeare quotes, on the other hand, had me audibly laughing in my cubicle.
-
Sorry, I tried to resist, but it just kept eating at me:
@snoofle said:
Since I'm the resident threading expert (at least around here)
That is what "resident" means, yes ;-)
-
In the same vein...
I had written a bit of code to display the status of various servers and plant machinery on a web page. It was mostly a simple affair, ping some machines, check if the FTP server was running, that the ISDN line was up, etc. The last bit, the plant machinery, was a pain. Some of them were 1970's relics, and the only info they gave out was a series of lights on their panel. I solved that with a bunch of PCs from the scrap heap and a mix of DACs made from Radio Shack parts, DPST relays, phototransistors, and black and white Logitech cameras pointed at the panels. It worked, but the fastest you could update some of the info with known fresh stuff was every 15-20 seconds. Once a minute was fine, I was told. After leaving them a pile of spare parts and documentation, I took my check and departed.
A year or so later, I get a call. Some of the plant machinery wasn't reporting in. I take a look remotely, and someone had mucked with the config file so that it tried to update every five seconds. Reverted the change, explained what happened, and sent a bill.
Two weeks go by, and in comes a second call. Nothing was working, and it had taken down their fileserver too! This time I drove over to have a look and smack the hand of whomever broke it.
Someone's nephew had been in to redesign the company webpage, and thought it would be cool to snazzify the status page and split it onto different versions. One for management, showing everything with uptime graphs, one for plant employees, showing a red/yellow/green board readable across the room, one for customers, complete with a little diagram of the plant and animated arrows showing the expected movement of things on the plant floor, and one each for dumbphones and Palm devices. It looked pretty damned spiff for the early naughties.
The goof? After breaking my status service once by making it poll too fast, and then discovering he didn't understand how I was generating the HTML with a mix of C and shell script, he'd gone with plan B; scrape the old web page and turn it into something his new code could read easily. But he was doing it five times, once for each version, keeping super-insane logs (Opening page for scrape at $time! Scrape successful! Retrieved $bytes in $seconds at $time! Mangling the scrape at $time!, etc), and doing it as fast as he possibly could. He was swamping the poor 486 that ran the status service and had filled the file-server with garbage logs.
At least his java was readable when it came time to fix it.
-
@snoofle said:
Since I'm the resident threading expert (at least around here)
I suspect that to achieve a relative level of "Expert" at your place of work, you would need to know what a thread is. Or, maybe just how to spell "Thread". If you actually know how to use them, that elevates you to somewhere north of Godhood.
-
@blakeyrat said:
@DaveK said:
You're not very observant, but I guess that's only to be expected when you go for sheer post quantity over quality. Cassidy's post links to an earlier post in the thread, and is thus not recursive. My post links to itself, and therefore illustrates recursion better.@Cassidy said:
@SteamBoat said:
That's not really it. You can actually learn more about recursion here.It's kind of like recursion. To understand recursion you have to, uh, well, there's that calling stuff and, you know
You can learn more about recursion here.Here's a puzzler: which joke is least funny? This "link to the same thread and call it a manual on recursion" joke? (Bonus points for making the EXACT SAME "JOKE" twice in the same thread.)
-
@DaveK said:
@blakeyrat said:
@DaveK said:
You're not very observant, but I guess that's only to be expected when you go for sheer post quantity over quality. Cassidy's post links to an earlier post in the thread, and is thus not recursive. My post links to itself, and therefore illustrates recursion better.@Cassidy said:
@SteamBoat said:
That's not really it. You can actually learn more about recursion here.It's kind of like recursion. To understand recursion you have to, uh, well, there's that calling stuff and, you know
You can learn more about recursion here.Here's a puzzler: which joke is least funny? This "link to the same thread and call it a manual on recursion" joke? (Bonus points for making the EXACT SAME "JOKE" twice in the same thread.)
How's this for observant: we get the joke, it's extremely lame, and no matter which post you refer to, we'll still get it, and it will still be at least as lame as if you'd referred to any other post in existence.
-
@DaveK said:
Cassidy's post links to an earlier post in the thread, and is thus not recursive. My post links to itself, and therefore illustrates recursion better.
And both are predictable and extremely unfunny. (Actually, as far as jokes go, a predictable punchline is basically the definition of "unfunny".) That's what I was commenting on. I'm sure the pedantic dickweeds appreciate that your "joke" meets the pedantic dickweed acceptance criteria; I only care that it's a fucking awful joke.
-
@DaveK said:
My post links to itself, and therefore illustrates recursion better.
Your post links to itself, and therefore illustrates an infinite loop. Good day sir.
-
@Zylon said:
Nope, that's infinite recursion. Please read and follow the instructions in this sentence.@DaveK said:
My post links to itself, and therefore illustrates recursion better.
Your post links to itself, and therefore illustrates an infinite loop. Good day sir.
-
-
@blakeyrat said:
@DaveK said:
You're not very observant, but I'm surprised that you don't even pay attention to your own posts. (Actually, I suppose that's also only to be expected when you go for sheer post quantity over quality.)Cassidy's post links to an earlier post in the thread, and is thus not recursive. My post links to itself, and therefore illustrates recursion better.
And both are predictable and extremely unfunny. (Actually, as far as jokes go, a predictable punchline is basically the definition of "unfunny".) That's what I was commenting on.
@blakeyrat said:
(Bonus points for making the EXACT SAME "JOKE" twice in the same thread.)
That's what I was commenting on. I wasn't "making the exact same joke", I was correcting a joke that, regardless how unfunny you find it, had been mis-told.Now, tell me again how unfunny you find it, and watch me care.