But it's on the queue



  • From the same Offshore developers that brought you this...

    Most of the time, our systems use the DB to send "messages" to each other. Once in a while, someone has used an actual queue to send messages. At my insistence, since they designed no mechanism to handle a queue dying event, the queue configurations were adjusted to be disk-backed. In many cases, there are potentially millions of messages sitting on a queue for hours at a time (batch systems talking to each other).

    This particular piece of brilliance puts a message on the queue at normal priority. Then at some time later, a user can specify: increase priority of some subset of messages.

    To do this, the messages are pulled off the queue (the sender reads the same queue to which they just posted), and reposted to the same queue with a higher priority.

    Then, just to make it interesting, this is repeated three times, each time bumping the priority by 1 (net increase of 4).

    Q. Why not just bump it by 4? A. Because it seemed like a big bump and we wanted to get all the messages in the subset processed (increased in priority) before any of them got read from the queue.

    Q. The queue is disk backed in what is essentially a random access file; doing this leaves lots of holes in the middle of the queue, and causes the disk space to ratchet up as the same messages keep getting re-appended to the back of the queue. Why? A. Disk space is cheap. Um, yeah, but the performance kind of eats the system as the file keeps getting reshuffled.

    Q. Since the target system already has two input queues (normal and high priority), why not just pull it off the normal queue and dump it on the high priority queue? Why jam it back onto the same queue in the first place, which causes the queueing system to keep scanning the queue for higher priority messages? A. We didn't think of it.

    And not a single line of comments anywhere.

     



  • Don't half the systems you work on flip a shit if things come down the pipe too quickly?

    Is there a reason a queue was chosen other than MSMQ is built into Windows? Assuming this is running on a Windows box.



  • Possible I'm TRWTF here, but presumably the priority is an int/byte/whatever in each record. If all you're doing is bumping up the priority, why not overwrite that int/byte/whatever? Or just implement one queue per priority level. Some form of locking will be needed in either case; the latter would seem (to my naive analysis) to cause fewer collisions.


  • ♿ (Parody)

    @Ibix said:

    If all you're doing is bumping up the priority, why not overwrite that int/byte/whatever? Or just implement one queue per priority level.

    "We didn't think of that."



  •  Basically... that's not a queue, just a list of items.



  • @snoofle said:

    And not a single line of comments anywhere.


    Not even this?

    // End of file


  • @bgodot said:

     Basically... that's not a queue, just a list of items.

     

    Don't make me put you on my queue for pedantic dickweeds.

     



  •  Isn't that more of a Stack?


Log in to reply