Name my class



  • I need more halp! And this time, it's not just about how much I suck.

    I'm making a class that sort of acts like a funnel. You put things on one end and those same things come out on the other. Except, rate-limited, so downstream won't get overwhelmed.

    For example, this minute I can submit 15 updates in and it will send only the last one submitted out (presuming I configured it to cache things for 1 minute).

    But later, it might not get any updates, so it won't emit any events.

    I know how to code this, no problem. I already did it a few times.

    But now I want to generalize it into its own thing, so... how do I name it?

    Is there an official name for this? Like a known design pattern or something?



  • I think I'm probably misunderstanding, but it sounds like some sort of buffer. Can you describe it in more detail?



  • Throttled something something?

    ThrottledEventProxy?



  • Rate-limiting?
    Throttling?

    When mucking about around telco industries, I've heard this kind of naming.

    Also, it's not a design pattern. Design is strategical, and this is more of an implementation thing.



  • Also, in filesystems world, it's called flushing.

    You either flush at regular intervals or when your queue is overfull, whatever comes first.



  • Buffer will pass through all the data it receives. This is sort of like a rate-limited buffer.

    Let's get concrete: I am keeping track of page click stats in a huge network and want to occasionally send update through socket.io to frontend. I keep my data like this:

    var stats = [{
    	page: "/home",
    	clicks: 1
    }, {
    	page: "/home/blagh",
    	clicks: 0
    }]
    

    So I'm receiving a constant stream of updates, but I don't want to overwhelm the browser with each click. Instead, what I want to do is this:

    var blagh = new HowDoINameThis({
    	rateLimitTo: 1000 * 30 // 30 seconds
    });
    
    blagh.on('data', function (stats) {
    	io.emit("stats", stats);
    });
    
    //...
    
    function onClick(page) {
    	var statOb = findStat(stats, page);
    	statOb.clicks++;
    	blagh.submit(stats);
    }
    


  • @MLo said:

    Throttled something something?

    ThrottledEventProxy?

    Yeah, throttled is definitely the word for this.

    ThrottledSieve?


  • SockDev

    @cartman82 said:

    I'm making a class that sort of acts like a funnel. You put things on one end and those same things come out on the other. Except, rate-limited, so downstream won't get overwhelmed.

    hmmm....

    SockProxy

    :stuck_out_tongue:

    as if i would suggest something else!



  • @cartman82 said:

    Is there an official name for this?

    I would have called this a queue. A rate limited queue. Stuff comes in, waits for its turn and then goes out.



  • But not all. Only the latest during the current buffering time frame.



  • @cartman82 said:

    For example, this minute I can submit 15 updates in and it will send only the last one submitted out (presuming I configured it to cache things for 1 minute)

    What happens to the other 14 updates? Are they sent out one by one every minute?


  • SockDev

    @accalia said:

    SockProxy

    So, how long before Sock* becomes a TDWTF meme?



  • I've heard this called a tarpit, I think it was in a mail server context.



  • Sometimes the line is empty and your turn happens right when you get there. But I'm an OR guy where we think a lot about queues.

    @cartman82 said:

    But not all. Only the latest during the current buffering time frame.

    I'm a little confused now, though. Are you possibly dropping stuff? Because originally you said:

    @cartman82 said:

    You put things on one end and those same things come out on the other.

    ...which I read as saying nothing gets dropped, just delayed.



  • @MLo said:

    What happens to the other 14 updates? Are they sent out one by one every minute?

    No they are dropped. The presumption is, all the updates coming in are newer and newer versions of the same thing.

    @Boomzilla, yeah, sorry. I wasn't clear.


  • SockDev

    @RaceProUK said:

    So, how long before Sock* becomes a TDWTF meme?

    it isn't already?



  • @cartman82 said:

    No they are dropped. The presumption is, all the updates coming in are newer and newer versions of the same thing.

    Oh, right. So it's a buffer that keeps the newest version of something, and every X time it sends that something on to something else if there was an update. Hmm...

    ThrottledObjectBufferStream ? :smiley:



  • @Boner said:

    I've heard this called a tarpit, I think it was in a mail server context.

    Google finds nothing. But there's this:



  • @MLo said:

    ThrottledObjectBufferStream

    Created through ThrottledObjectBufferStreamAbstractInjectionFactoryInterface?




  • kills Dumbledore

    Jeff.DoingItWrong.RateLimit.AllTheThings.TryAgainInOneDiscoMoinute



  • TimedEventPump?



  • I'm going with EventThrottler. Here's a quick fiddle sketch of what it'll do.

    function EventThrottler(defaultDelay) {
    	this._throttled = {};
    }
    
    EventThrottler.prototype.submit = function submit(channel, delay, data) {
    	if (arguments.length < 3) {
    		data = delay;
    		delay = defaultDelay;
    	}
    
    	if (!this._throttled[channel]) {
    		this._throttled[channel] = {
    			channel: channel,
    			timeout: setTimeout(this._onTimeout.bind(this, channel), delay);
    		};
    	}
    	this._throttled[channel].data = data;
    };
    
    EventThrottler.prototype._onTimeout = function _onTimeout(channel) {
    	var def = this._throttled[channel];
    	if (!def) {
    		return;
    	}
    
    	delete this._throttled[channel];
    	if (typeof(def.data) === "function") {
    		return def.data(this.emit.bind(this, channel));
    	}
    
    	return this.emit(channel, data);
    };
    

    I still have to plug it into node, test it, add a few bells & whistles etc.

    But I'm surprised there wasn't already a ready made pattern for this. I mean, this seems like a common type of problem to have, especially with socket.io stuff.



  • @cartman82 said:

    I mean, this seems like a common type of problem to have, especially with socket.io stuff.

    I'm not sure. It's unusual to drop all but one update.





  • That's a pretty good match actually...



  • @Buddy said:

    debounce

    That's the one, @Buddy!

    Switching name to Debouncer.

    I knew there was an official name for this.



  • AIUI
    debounce is within the producer of the messages.
    to me your class is a messageBuffer or a bufferedQeue
    (a buffer it's just a place to store the things generated by the producer until the consumer actually consumes them, or that's what i've been told)

    anyway, i sound like someone saying "see, that bikeshed color it's not purple, it's lilac"



  • I'd go for the word aggregate. As in, you're aggregating any messages that come in in the interval.



  • @PleegWat said:

    As in, you're aggregating any messages that come in in the interval.

    Not really though. He's discarding all but the final one. It's like a one element buffer.



  • @Jarry said:

    to me your class is a messageBuffer or a bufferedQeue(a buffer it's just a place to store the things generated by the producer until the consumer actually consumes them, or that's what i've been told)

    But not all messages are consumed.

    @PleegWat said:

    I'd go for the word aggregate. As in, you're aggregating any messages that come in in the interval.

    Aggregate feels like it should combine multiple things into one. This doesn't do that.

    I implemented it as Debouncer.
    That's a good unique name and it's already used in a popular js library.



  • @Jarry said:

    to me your class is a messageBuffer or a bufferedQeue

    Now that I understand what he's doing, it seems like there should be filter in the name somewhere.



  • Bottleneck



  • @cartman82 said:

    I implemented it as Debouncer.That's a good unique name and it's already used in a popular js library.

    It would also confuse the bejeesus out of me - a debouncer, to me, is something that filters out spurious updates, not rate-limits them.



  • I think it's a LIOO stack. It's a whole new paradigm.



  • I like the word "throttled" that people have been using. I would suggest: ThrottledEventQueue



  • Buy WebTrends, I used to work there, good people.



  • @CoyneTheDup said:

    I like the word "throttled" that people have been using. I would suggest: ThrottledEventQueue

    To me, queue implies FIFO semantics. That's why I proposed pump - in analogy to WinAPI message pump.



  • Isn't it what he is doing ? He accumulates updates for a while then serves the last. (His initial explanation is inaccurate)

    @Keith said:

    I think it's a LIOO stack. It's a whole new paradigm.

    this



  • @Jarry said:

    debounce is within the producer of the messages.
    Call it a Bouncer then. Like those guys preventing too many people from entering a club.

    As an aside, I like the sound of ThrottleGate, regardless of whether it is applicable.



  • @blakeyrat said:

    good people

    Not sure what to expect.



  • @cartman82 said:

    Is there an official name for this? Like a known design pattern or something?

    Flow control?



  • @xaade said:

    Not sure what to expect.

    They have a streaming web analytics product that runs over WebSockets, and is pretty damned slick.



  • LIFORD



  • I like liford.

    Vote for liford.



  • Wouldn't it be LILORD then? I assume the last update is the newest one, which is the one he wants to propagate.

    EDIT: no wait I am dumb. Where is my morning coffee?



  • Next time someone laughs when I say one of the hardest things in programming is naming components correctly, I'll point them to this thread.



  • @Eldelshell said:

    Next time someone laughs when I say one of the hardest things in programming is naming components correctly, I'll point them to this thread.

    Yup. And this is ABOVE AVERAGE as far as name ideas are concerned. A lot of good suggestions.

    Any day you don't have to go to thesaurus.com to name your shit is a good day.



  • @cartman82 said:

    Any day you don't have to go to thesaurus.com to name your shit is a good day.

    When in doubt, just name it Steve. It's a nice name.



  • it seems to behave like a Generic Cell Rate Algorithm with a small bucket. very small. And opposite of usual bucket which keep the first and dismiss the late.

    SingleNewestTokenBucket

    that, or going to thesaurus for traffic shapping and rate limit


Log in to reply
 

Looks like your connection to What the Daily WTF? was lost, please wait while we try to reconnect.