RSS WTF?



  • Caveat:  I'm not 100% sure that this is a WTF, but it's a bit convoluted and I thought I'd share it and ask if you think it's a WTF

    I wish I remembered more of the details.

    At my last job, there were two programmers:  me (writing Java programs for cell phones) and the other guy, let's call him Justin (web guy, creating things with RoR).

    There was a Ruby (not Rails) aspect of his project that sent out text messages when they needed to be sent.  He queued them up internally with an RSS feed, then checked that feed every minute or so, and when there was something in it, the messages got sent out with kannel.  Some of the messages originated from the database.  Others (like weather and traffic) originate from the NWS or local news websites.  

    I don't remember any more of the details, but this always sounded WTF-y to me.  Then I thought

    hey, the RSS creating and reading is probably done with a library in Ruby.  If it's really nothing more than API calls as opposed to writing his own parser for some arbitrary organization of data, is it really that much of a WTF?

    does this sound like a WTF to you?  Ask me questions about it.  Maybe they'll help me remember more of the details.



  • Not a wtf just for the reason you gave, its very easy to implement and makes it easy to add new sources when needed as long as those sources have a RSS feed.



  • Sounds like a failrly plain situation where you normalize the data before doing your magic. 



  • Actually, it sounds horribly convoluted.  For one, an XML document is probably one of the stupider ways to implement a message queue.  Second, RSS isn't even a very good protocol for these kinds of things because it has so few attributes that can be use.  It also doesn't matter if the original message source is RSS or whatever (as stratos suggested) because the messages still have to be enqueued for each user.  The best way to handle this would probably be something like a simple text file that lists each message in the order they are queued.  This avoids the problems of using a relational database as a message queue and avoids the bloat of RSS and its unsuitability for representing SMS messages.



  • @morbiuswilters said:

    The best way to handle this would probably be something like a simple text file that lists each message in the order they are queued.  This avoids the problems of using a relational database as a message queue and avoids the bloat of RSS and its unsuitability for representing SMS messages.
    It introduces potential concurrency issues, however.

    I'd generally use a database table, personally (assuming your app is already database driven, and in absence of a better queuing solution).  Handles the concurrency issues, and it's relatively simple to interact with. 



  • @belgariontheking said:

    At my last job, there were two programmers...

     

    It sounds like you were at a small shop...probably with no code review?  I don't think it's that big of a WTF if he wanted to try out the RSS thing and had to stretch its use past what it was designed for.  Obviously probably not the best idea, but if it handles the load of the system quick enough to please the company, why should he or anyone else care?  The small companies I have worked at have had the "it's good if it works and doesn't take forever" general attitude.



  • If it was the same service/machine receiving the messages, queuing them, hosting the RSS feed, polling the RSS feed, and sending out the messages, then yes, it seems a little ridiculous.

    I'm not sure if that's what's going on, though, and it's hard to tell by your explanation.  If the weather and traffic alerts were already RSS, then I can see it sort of making sense - why write your own message queue when you can use something off-the-shelf?  Just make a separate RSS feed for your own alerts and it's easy to combine everything into a single "queue".

    On the other hand, if the weather and traffic alerts were just text messages, and nothing was actually RSS at the "source", then it's just a pointless extra layer and there was really no need for any kind of message queue (just sent them out as they come in).

    So I guess the short answer is... it depends? 

     



  • @merreborn said:

    It introduces potential concurrency issues, however.

    I'd generally use a database table, personally (assuming your app is already database driven, and in absence of a better queuing solution).  Handles the concurrency issues, and it's relatively simple to interact with.

    Use a global mutex to prevent concurrency issues.  A database table makes a very poor FIFO queue. 



  • @Digitalbath said:

    I don't think it's that big of a WTF if he wanted to try out the RSS thing and had to stretch its use past what it was designed for.

    That is the whole point of this site: discussing contrived and inefficient solutions such as this.  It doesn't matter how small the shop was, somebody will have to maintain this beast someday and will likely say "WTF was this guy thinking?" when they see the RSS "solution". 



  • @Aaron said:

    If it was the same service/machine receiving the messages, queuing them, hosting the RSS feed, polling the RSS feed, and sending out the messages, then yes, it seems a little ridiculous.
    Yep.  We had one production (live) machine and one dev machine (staging).  They did everything for their respective environments, from database to apache to ruby to actually JIT-compiling the Java mobile app as it went out.  

    @Aaron said:

    On the other hand, if the weather and traffic alerts were just text messages, and nothing was actually RSS at the "source"
    The weather was done through a service (I believe it's called NOAH).  The traffic was (I believe) screen scraped.  A handful of others were typed in by the client through another webapp (hosted on the same box) and then sent out immediately, like the breaking news feed.  So yes, the programmer had to create an RSS feed for each feed, then the sending app polled those feeds.

     

    cuz I know there will be questions:

    @belgariontheking said:

    JIT-compiling the Java mobile app as it went out
    This sounds a bit WTF-ey but we needed a way for the app to know who the user was without the user typing in their phone number.  So I had it compile the phone number into a class when the user requested the download.  I tried to make it just a text file shipped along with the jar but that didn't work for some reason.  Asking a user on first run, then saving a file didn't work for some reason.  I think that Java couldn't guarantee that the file would be persistent.  Remember that with phones, you gotta program for the lowest common denominator, or else you're going to compile a different version for each phone that you want to run on.  That's true of any program, but for phones, the lowest common denominator is that much lower.



  • @merreborn said:

    I'd generally use a database table, personally (assuming your app is already database driven, and in absence of a better queuing solution).  Handles the concurrency issues, and it's relatively simple to interact with. 
     

    I thought so too, but then I noticed btk wrote "arbitrary data structure" or something like that ...

    So lets get at it :

    1. write a parser for each incoming stream

    2. producer stores the data in a db and normalises it in the process

    --- 

    3. consumer reads the data, denormalises it

    4. formats it for output

     

    or

     

    1. producer copies the data as it comes from different sources to the rss feed

    -- 

    2. consumer reads the data

    3. formats it for output 

     

    i can see why he chose the second thing, skips parsing, simpler producer, moves the parsing to the consumer ...

    not that i agree with him ... thought i'd never write this, but i have to agree with morb - i would parse the input data and use a simple text file for storage or even shared memory ...

    as for concurrency issues, producer consumer pattern is implemented 100 times before in every language available ...

     




  • @morbiuswilters said:

    Second, RSS isn't even a very good protocol for these kinds of things because it has so few attributes that can be use.
     

    Unless, of course, the RSS content is actually just another XML document! 

    Hmmm, I wonder if the aggregated RSS feed contained the component RSS feeds in such a fashion?



  • @PhillS said:

    I wonder if the aggregated RSS feed contained the component RSS feeds in such a fashion
    IIRC, there was no "aggregated RSS feed."  Each source (weather, traffic, sports) had its own internal feed.


Log in to reply