Not so real time, Real Time



  • We have an app which collects and collates data from various sources when certain events happen (Worse description ever, I know, but any more detail than that gives away what I'm talking about). We store this data in <warning type="marketing speak ahead">normalised data structure</warning> (aka a database), which is then collated and summarised into an analytics database for reporting purposes once every 15 minutes. We even supply some industry standard reports that are updated automatically via this process. For anyone who wants it, we also offer an API which can be used to receive events via a message queue in real (ish) time (or as long as it takes us to receive the event and send it on). Pretty standard stuff.

    One of our new customers loved the system. After they'd had it for 8 months, they come back to us and ask us to "optimise" the real time feed, because its not fast enough for a new project of theirs... they have to know the moment anything happens for their new "management dashboard". So we work hard at it, for 6 months, and we manage to make some headway - with some acceptable gains here and there. They go away happy (happy customers happen to us from time to time), and implement this new fancy dashboard internally... we never hear of any complains or problems.

     

    Fast forward 6 months. Me and a colleague are doing a site visit to the same customer for an unrelated project, and while we're there I ask if we could see this new dashboard. Calls are made, and the project manager for the dashboard appears as if by magic. Proud as punch, he offers to take us on a personal tour of "management HQ", where the dashboard is used. Ok we say, and in we go. 

     Its actually quite impressive... looks like something from a movie, lots of people with their own workstations sitting in rows, facing a larger screen on the wall (or in this case, in a darkened room with a projected dashboard on the wall). It all looks quite nice. The data seems to be working, some of it even looks useful, but we notice that it doesn't update very often. We know events occur pretty fast in this environment, things change ALOT at all times of the day - so if this thing is being driven by our Real Time data, it should be changing and updating all the time. Side ways glances at a colleague confirm he's thinking the same thing... unable to help ourselves, we ask about it..

    Me : "So this is all driven by our RT Feed ?"
    PM : "Yeap, looks good doesn't"
    ME : "Yes, very impressive, its just... well, with an RT Feed I would have expected the data to be updating a little more... regularly ?"
    PM : "It does, this stuff updates every 15 minutes!! The old one only updated once an hour!! we're ahead of the curve"
    ME : "But you had us working on the RT Feed for 6 months to improve its speed... and yet you're not using it"
    PM : "Of course we are. We take your RT Feed and we store the events from it in a normalised data structure, which we then report on every 15 minutes to create the Dashboard you see here"
    ME : "oh, ok..."
    PM : "Of course, thanks to the RT Feed, we update faster than before"
    ME : "But you only update the screens every 15 minutes"

    The room fell silent, apart from the whirling sound of the projector's fans... All eyes turn to look at us, as people are sit by a sudden realisation.

    It turns out, Management HQ was supposed to use our original system. When the PM came across the RT Feed in our documentation, he decided it would be make things so much **faster** for his project. Their project ended up re-implementing everything our product already did, on top of our product, and right up until our visit no one had asked "why".




  • @deathy said:

    It turns out, Management HQ was supposed to use our original system. When the PM came across the RT Feed in our documentation, he decided it would be make things so much faster for his project.

    But... isn't he the one who asked for it to be faster in the first place? How was he just "discovering" it in the documentation? Your story lost me a little bit.


  • Trolleybus Mechanic

     My prediction: 75% of TDWTF readers will see this thread in their email notification, see the random capitalization, associate "real time" with a certain desktop video player, and never click through to read this story.

     



  • @Lorne Kates said:

     My prediction: 75% of TDWTF readers will see this thread in their email notification, see the random capitalization, associate "real time" with a certain desktop video player, and never click through to read this story.

     


    My prediction: 75% of TDWTF readers will lose interest in this thread OVER THREE YEARS before it was posted.



  • @Ben L. said:

    @Lorne Kates said:
    My prediction: 75% of TDWTF readers will see this thread in their email notification, see the random capitalization, associate "real time" with a certain desktop video player, and never click through to read this story.

     

    My prediction: 75% of TDWTF readers will lose interest in this thread OVER THREE YEARS before it was posted.
    Only in New York City.

     



  • @El_Heffe said:

    @Ben L. said:

    @Lorne Kates said:
    My prediction: 75% of TDWTF readers will see this thread in their email notification, see the random capitalization, associate "real time" with a certain desktop video player, and never click through to read this story.

     


    My prediction: 75% of TDWTF readers will lose interest in this thread OVER THREE YEARS before it was posted.
    Only in New York City.

     


    Unless they install Buildmaster. Builds go so much faster since we implemented BuildMaster. It's an amazing product from Indeo



  • @mikeTheLiar said:

    It's an amazing product from Indeo
     



  • @mikeTheLiar said:

    Unless they install Buildmaster. Builds go so much faster since we implemented BuildMaster. It's an amazing product from Indeo India
    l2spell, nub



  • @mikeTheLiar said:

    @El_Heffe said:

    @Ben L. said:

    @Lorne Kates said:
    My prediction: 75% of TDWTF readers will see this thread in their email notification, see the random capitalization, associate "real time" with a certain desktop video player, and never click through to read this story.

     


    My prediction: 75% of TDWTF readers will lose interest in this thread OVER THREE YEARS before it was posted.
    Only in New York City.

     


    Unless they install Buildmaster. Builds go so much faster since we implemented BuildMaster. It's an amazing product from Indeo

    Goddammit, I'm so behind on my memes..

    Maybe, uh, he should have printed the thread out and put it on a wooden table and taken a photograph and use that for the title! Ha ha!




    Goddammit you guys..



  • @morbiuswilters said:

    Maybe, uh, he should have printed the thread out and put it on a wooden table and taken a photograph and use that for the title! Ha ha!
     

    We got three minimized Excel windows, which we should have used. In one of them, there's a joke. The others contain a bitcoin.


  • Trolleybus Mechanic

    @dhromed said:

    We got three minimized Excel windows, which we should have used. In one of them, there's a joke. The others contain a bitcoin.
     

    I see you're trying to start a flamewar...

                             /






  • @flabdablet said:



    Damn, so close to winning the jackpot..



  • @morbiuswilters said:

    @flabdablet said:


    Damn, so close to winning the jackpot..

    If the lemon is behind goat 3 and I open goat 1, should you switch or stay?



  • @Ben L. said:

    If the lemon is behind goat 3 and I open goat 1, should you switch or stay?

    If you start "opening" goats, I'm definitely not going to stay..



  •  To be clear, he read about the RT Feed in the documentation, and decided it would suit his project perfectly - which is why he asked us to speed it up. How he managed to read up to that point and not realise we did everything his project did... I'll never know.



  • @morbiuswilters said:

    @flabdablet said:


    Damn, so close to winning the jackpot..





  • @morbiuswilters said:

    @Ben L. said:
    If the lemon is behind goat 3 and I open goat 1, should you switch or stay?
    If you start "opening" goats, I'm definitely not going to stay..
    Why? It's a time-honored tradition.



  • @deathy said:

    So we work hard at it, for 6 months, and we manage to make some headway

    Who paid for this work? Hopefully the customer -- after all, everyone else is happy with the performance; and there is no reason to work hard for 6 months on something nobody cares about.

    And for 6 months; presumably the customer kept saying "no, not fast enough yet..." until finally he said "OK, that's fast enough".

    If the customer paid for the work, and signed off on it, why do you care what he does with the results?

    If, on the other hand, you did this work out of the goodness of your heart -- well, thats a WTF. 6 months of tweaking, when you could have been working on another product that actually might produce an additional revenue stream?


Log in to reply