To be clear, he read about the RT Feed in the documentation, and decided it would suit his project perfectly - which is why he asked us to speed it up. How he managed to read up to that point and not realise we did everything his project did... I'll never know.
deathy
@deathy
Best posts made by deathy
Latest posts made by deathy
-
RE: Not so real time, Real Time
-
Not so real time, Real Time
We have an app which collects and collates data from various sources when certain events happen (Worse description ever, I know, but any more detail than that gives away what I'm talking about). We store this data in <warning type="marketing speak ahead">normalised data structure</warning> (aka a database), which is then collated and summarised into an analytics database for reporting purposes once every 15 minutes. We even supply some industry standard reports that are updated automatically via this process. For anyone who wants it, we also offer an API which can be used to receive events via a message queue in real (ish) time (or as long as it takes us to receive the event and send it on). Pretty standard stuff.
One of our new customers loved the system. After they'd had it for 8 months, they come back to us and ask us to "optimise" the real time feed, because its not fast enough for a new project of theirs... they have to know the moment anything happens for their new "management dashboard". So we work hard at it, for 6 months, and we manage to make some headway - with some acceptable gains here and there. They go away happy (happy customers happen to us from time to time), and implement this new fancy dashboard internally... we never hear of any complains or problems.
Fast forward 6 months. Me and a colleague are doing a site visit to the same customer for an unrelated project, and while we're there I ask if we could see this new dashboard. Calls are made, and the project manager for the dashboard appears as if by magic. Proud as punch, he offers to take us on a personal tour of "management HQ", where the dashboard is used. Ok we say, and in we go.
Its actually quite impressive... looks like something from a movie, lots of people with their own workstations sitting in rows, facing a larger screen on the wall (or in this case, in a darkened room with a projected dashboard on the wall). It all looks quite nice. The data seems to be working, some of it even looks useful, but we notice that it doesn't update very often. We know events occur pretty fast in this environment, things change ALOT at all times of the day - so if this thing is being driven by our Real Time data, it should be changing and updating all the time. Side ways glances at a colleague confirm he's thinking the same thing... unable to help ourselves, we ask about it..
Me : "So this is all driven by our RT Feed ?"
PM : "Yeap, looks good doesn't"
ME : "Yes, very impressive, its just... well, with an RT Feed I would have expected the data to be updating a little more... regularly ?"
PM : "It does, this stuff updates every 15 minutes!! The old one only updated once an hour!! we're ahead of the curve"
ME : "But you had us working on the RT Feed for 6 months to improve its speed... and yet you're not using it"
PM : "Of course we are. We take your RT Feed and we store the events from it in a normalised data structure, which we then report on every 15 minutes to create the Dashboard you see here"
ME : "oh, ok..."
PM : "Of course, thanks to the RT Feed, we update faster than before"
ME : "But you only update the screens every 15 minutes"
The room fell silent, apart from the whirling sound of the projector's fans... All eyes turn to look at us, as people are sit by a sudden realisation.
It turns out, Management HQ was supposed to use our original system. When the PM came across the RT Feed in our documentation, he decided it would be make things so much **faster** for his project. Their project ended up re-implementing everything our product already did, on top of our product, and right up until our visit no one had asked "why". -
The 2 Field Solution
So, for the last 3 years I've been responsible for a data collection app. The server-side of this app is looked after by another team.
When the data app was relatively new (read, not finished - and its still not) we were pushed by management into a customer project. The server side guys rushed out a web service for us to talk to, and in the space of a few days we went from "not ready" to "barely-ready-but-not-tested".
3 years later, that web service still exists. And its still the main contact point between my app and the server. And its not changed in all that time - which means its incapable of gathering 90% of the data the app generates.
At managements request, the app has improved capabilities but still has no way of sending these updated values to the server for reporting. Not that it matters; the reporting guys are yet another team, and they "customised" the database (without input from anyone else), which means the web service doesn't populate the database correctly for the reporting system. So the reporting system ignores it... so... yeap, live data isn't reported on. Until the customer sends it back to us for "analysis", at which point we run a SQL script to update it.
Despite constant attempts, I cannot get it though to Management that we need to concentrate on this "product" and finish the app, the web service and the reporting.
Anyway. Fast forward to this morning. I've just got out of a meeting where the server-side team (with a straight face) decided that rather then re-design the whole web service (which we need them to do) they'll add two fields for one customer project. Will they update the code that writes to the database - no, its out of scope for this project. So...we can record two additional fields. We can't report on them. And they won't fold the changes into their installer (written by me, because they insisted they didn't need one). So the install guys will have to install the server, "apply" this update... and it'll still have all of the major problems it had before.
3 years of "product development", and we're... nowhere. And we're continuing to go no where fast. We move from customer to customer, implementing (badly) custom code while exactly zero product development goes on in the background. We have nothing. How this place gains customers amazes me, but it does and it pays the bills...
-
RE: But I closed it...
Ha, the old "but i did close it" line.
I believe someone once asked "so what if it leaks" on the titanic... that turned out well.
-
RE: Quick and Effortless
@Cassidy said:
@deathy said:
Mainly because all configuration changes require numerous server reboots (for various reasons outside of our control - thats not the WTF, but it should be).
I absolutely hate changes that require reboots.
(cue tale of a hosting company that rebooted my root server after plugging in an external USB disk. There was a reason I paid for the USB option, rather than the internal option. Fuckwits.)
If only the reboots were because of software... Its more because the configuration changes are applied to hardware devices which are fussy at best - but thats a WTF for another time.
-
RE: Quick and Effortless
@deathy said:
mod: changed PRE tags to P, you silly user you! -dh
I beg to differ - community server sucks. On a positive, thanks for the edit.
@tchize said:
@deathy said:
You can stop at that point and fire him :)2 Weeks later he checked in the first version.
To have the power...
@dargor17 said:
@Rick said:
the first process writes a file and the second process installs this file.
My guess too, but it still seems so convoluted and error-prone that I can't imagine why he would want to do that
I have no idea why he did this way, to be entirely honest and to save my sanity I never looked at the code. Java isn't one of my major strong suits, but last time I messed with it IPC wasn't exactly easy.The current thinking among the rest of us is that he tried several different ways, which is why it took so long, and settled on this method purely because people were starting to ask what the hell was taking so long. It raises the question of why. But we're all of the opinion our heads would explode if he explained it to us.
The guy is well known for the convoluted format of his work, but since he's a subject-matter-expert for one particular project he's considered "important" to those at the top. Thankfully, he doesn't get allowed anywhere near the important stuff by the rest of us.
To be entirely fair to him, he does make the most awesome cup of coffee I've ever had. And his missus cooks the most amazing cakes.
-
Quick and Effortless
So... our data collection system is quite a pain to configure. Mainly because all configuration changes require numerous server reboots (for various reasons outside of our control - thats not the WTF, but it should be).
One of the simplest (and more common) configuration changes people want to make is changing the database settings. Normally the users have to edit a file, which contains a grand total of 1 line;
jdbc:mysql://localhost:3306/thedb
This confuses users. We'll ignore the fact that only system admin's should be messing with this setting, but still.
So one of my colleagues was assigned to create a config tool. We all had a meeting about it, where we discussed making it nice and easy with checkboxes, etc... He started work.
2 Weeks later he checked in the first version. Strike One.
It consisted of a Java App which opens the file, and dumps its contents into an edit box. There are no other controls on the screen. Strike Two.
You have to run the Java App a second time with a command-line to save the changes in the first app. Strike Three.
He's been relieved of that project. It was handed to the most junior member of the team. A day and a half later the junior checked in version 1, with it all done. In version 2.0 he expects to have fixed one spelling mistake and added a graphic.
mod: changed PRE tags to P, you silly user you! -dh
-
RE: I can write threaded code
@snoofle said:
A long time ago the designers of our application attempted to make it multi threaded, and failed. So the code is in place, but the number of threads is configured as one.
I was asked to make our single threaded application multithreaded so as to reduce the footprint. That is, instead of 20 instances each with a multi GB cache footprint, run one instance with multiple threads and share a cache.
Then I got into the code.
First, they used Spring to create the db connection pool. No problem. Then they wrote their own connection pool manager. And they wrote it incorrectly: it leaked connections. So close.
Next, they wrote "beans" (cough) - with every method and local variable public and static (OO anyone?)
In the code, they load the beans via Spring (getBean) in one place, and simply instantiate them (via new) everywhere else (consistency anyone?)
Since there were problems with the original threaded code, the developer put in strategically placed sleep's in order to keep the program from tripping over itself. And the sleep's were cpu-speed dependent, so when they upgraded hardware two years ago, the timing changed, and strange bugs cropped up. SVN blame says the only changes were to the sleep-times.
Finally, there were the singleton objects with unsynchronized public non volatile static variables that were supposedly thread-safe. Of course every thread would just trample the values set by every other thread.
When I asked the developer who wrote it how it was supposed to be thread safe given what he had written, he replied: "I can write threaded code, but for some reason it just didn't work."
I then attempted to show him why it didn't work, but he insisted that "the JVM is supposed to keep all that stuff straight".
*sigh*
Thats actually worse than at our place... where a pool of threads all operate on, and share numerous resources which basically make the whole multi-threaded effort pointless. Yet support still tell customers to increase the thread count as step 1 in the process for solving performance issues. It doesn't help, but it makes customers feel special and like we're listening (And makes support feel like they know something - which they don't).
-
Source Control is for the Weak
Hi All,
Following on from my last post (see here), I have a confession to make.
I didn't use Source Control for the product re-write. Now before everyone goes and screams at me (like you should), I didn't use it because I'd never heard of it until I was half done and it was always identified as a team tool. With a team of 1, I decided it wasn't worth the effort to implement (and will happily admit to being wrong).
Fast forward a couple of months from there, and I eventually got to understand what else source control provides. So I did my research, considered things carefully, and went with what the university used; CVS. It was fun getting used to using it, changing my work habits, realising how powerful the tool could be, etc...
Fast forward another few months, the product is done and its time for me to move on to another project. So management bring in another developer, this time one with alot of experience. He will be the maintenance guy for the product, and he will work with me for a few months learn the product and the code and then take over.
First day on the job, we go though the usual things, I get him setup to download a copy of the code, etc... He see's me using CVS and asks me what thats for. More then a little supprised, I explain its our source control system and I use it to keep track of versions, and now there are two of us on the team, to share code between us. He stutters for abit, then shouts "I'm not using that... waste of time, causes nothing but problems". I ask him what he means, nervous as I was that in this open plan office management heard every bit of his outburst. And he explains "its a bloody stupid concept, if I delete a file locally that stupid thing will undelete it and put it back - makes working impossible". I try to explain why that happened only to be met with ... "no, you don't understand. Source Control isn't useful. Nobody in the industry uses it, so why should I? Stick it on a network drive, and be done with it" with the eyes of the whole office boring in the back of my head, he finishes his outburst with "thats how we did it at my old place".
It took me a while to get my breath back, and I could only think to ask one question; where was your old place? His reply "The Government".
Turns out Mr Happy has worked for a short time in the local government offices (read : Council), developing an app to access a database provided by central government. This app was his "baby" but it didn't do "Windows", it only did "men's work; DOS". So he was booted when everyone switched to Windows, and thats how he came to us. He stayed for 4 months. And in that time he successfully managed to control his rage over Source Control, his anger over having a youngster as his boss, and finally his annoyance over not having a parking space in the office carpark. In all faireness, he did fix a major customer problem at the time - pressing F1 to get help didn't work.
Later,
Deathy
-
RE: The Reporting Nightmare
@lolwtf said:
Sounds like they just automated the task they were doing by hand, without bothering to wonder if perhaps there were more efficient ways... or assuming that because the only way they knew to dump a window's contents to a bitmap involved pasting into word, that was the only way it could be done.
Closing and restarting Word for every screenshot, now that's insane. Did they leave any hint why? Did you try making it re-use the existing instance to see if everything went (even more) horribly wrong?You assume they left alot of comments.
I will say they were extremely good at inserting a copyright notice at the top of every unit/module.But then you would be too, if you had an automated tool for doing that.
I did consider just re-using the same instance of word, but remember that management wanted the word dependency gone. So I focused my goal on that first. From what I remember, they restarted word every time because they left word in an "unknown" state - they didn't know what they'd done and didn't want to find out.