Name my class
-
I need more halp! And this time, it's not just about how much I suck.
I'm making a class that sort of acts like a funnel. You put things on one end and those same things come out on the other. Except, rate-limited, so downstream won't get overwhelmed.
For example, this minute I can submit 15 updates in and it will send only the last one submitted out (presuming I configured it to cache things for 1 minute).
But later, it might not get any updates, so it won't emit any events.
I know how to code this, no problem. I already did it a few times.
But now I want to generalize it into its own thing, so... how do I name it?
Is there an official name for this? Like a known design pattern or something?
-
I think I'm probably misunderstanding, but it sounds like some sort of buffer. Can you describe it in more detail?
-
Throttled something something?
ThrottledEventProxy?
-
Rate-limiting?
Throttling?When mucking about around telco industries, I've heard this kind of naming.
Also, it's not a design pattern. Design is strategical, and this is more of an implementation thing.
-
Also, in filesystems world, it's called flushing.
You either flush at regular intervals or when your queue is overfull, whatever comes first.
-
Buffer will pass through all the data it receives. This is sort of like a rate-limited buffer.
Let's get concrete: I am keeping track of page click stats in a huge network and want to occasionally send update through socket.io to frontend. I keep my data like this:
var stats = [{ page: "/home", clicks: 1 }, { page: "/home/blagh", clicks: 0 }]
So I'm receiving a constant stream of updates, but I don't want to overwhelm the browser with each click. Instead, what I want to do is this:
var blagh = new HowDoINameThis({ rateLimitTo: 1000 * 30 // 30 seconds }); blagh.on('data', function (stats) { io.emit("stats", stats); }); //... function onClick(page) { var statOb = findStat(stats, page); statOb.clicks++; blagh.submit(stats); }
-
Throttled something something?
ThrottledEventProxy?
Yeah, throttled is definitely the word for this.
ThrottledSieve
?
-
I'm making a class that sort of acts like a funnel. You put things on one end and those same things come out on the other. Except, rate-limited, so downstream won't get overwhelmed.
hmmm....
SockProxy
as if i would suggest something else!
-
Is there an official name for this?
I would have called this a queue. A rate limited queue. Stuff comes in, waits for its turn and then goes out.
-
But not all. Only the latest during the current buffering time frame.
-
For example, this minute I can submit 15 updates in and it will send only the last one submitted out (presuming I configured it to cache things for 1 minute)
What happens to the other 14 updates? Are they sent out one by one every minute?
-
-
I've heard this called a tarpit, I think it was in a mail server context.
-
Sometimes the line is empty and your turn happens right when you get there. But I'm an OR guy where we think a lot about queues.
But not all. Only the latest during the current buffering time frame.
I'm a little confused now, though. Are you possibly dropping stuff? Because originally you said:
You put things on one end and those same things come out on the other.
...which I read as saying nothing gets dropped, just delayed.
-
What happens to the other 14 updates? Are they sent out one by one every minute?
No they are dropped. The presumption is, all the updates coming in are newer and newer versions of the same thing.
@Boomzilla, yeah, sorry. I wasn't clear.
-
-
No they are dropped. The presumption is, all the updates coming in are newer and newer versions of the same thing.
Oh, right. So it's a buffer that keeps the newest version of something, and every X time it sends that something on to something else if there was an update. Hmm...
ThrottledObjectBufferStream
?
-
I've heard this called a tarpit, I think it was in a mail server context.
Google finds nothing. But there's this:
-
ThrottledObjectBufferStream
Created through
ThrottledObjectBufferStreamAbstractInjectionFactoryInterface
?
-
http://en.wikipedia.org/wiki/Tarpit_(networking)
Not applicable here anyway.
-
Jeff.DoingItWrong.RateLimit.AllTheThings.TryAgainInOneDiscoMoinute
-
TimedEventPump?
-
I'm going with
EventThrottler
. Here's a quick fiddle sketch of what it'll do.function EventThrottler(defaultDelay) { this._throttled = {}; } EventThrottler.prototype.submit = function submit(channel, delay, data) { if (arguments.length < 3) { data = delay; delay = defaultDelay; } if (!this._throttled[channel]) { this._throttled[channel] = { channel: channel, timeout: setTimeout(this._onTimeout.bind(this, channel), delay); }; } this._throttled[channel].data = data; }; EventThrottler.prototype._onTimeout = function _onTimeout(channel) { var def = this._throttled[channel]; if (!def) { return; } delete this._throttled[channel]; if (typeof(def.data) === "function") { return def.data(this.emit.bind(this, channel)); } return this.emit(channel, data); };
I still have to plug it into node, test it, add a few bells & whistles etc.
But I'm surprised there wasn't already a ready made pattern for this. I mean, this seems like a common type of problem to have, especially with socket.io stuff.
-
I mean, this seems like a common type of problem to have, especially with socket.io stuff.
I'm not sure. It's unusual to drop all but one update.
-
-
That's a pretty good match actually...
-
debounce
That's the one, @Buddy!
Switching name to
Debouncer
.I knew there was an official name for this.
-
AIUI
debounce is within the producer of the messages.
to me your class is a messageBuffer or a bufferedQeue
(a buffer it's just a place to store the things generated by the producer until the consumer actually consumes them, or that's what i've been told)anyway, i sound like someone saying "see, that bikeshed color it's not purple, it's lilac"
-
I'd go for the word
aggregate
. As in, you're aggregating any messages that come in in the interval.
-
As in, you're aggregating any messages that come in in the interval.
Not really though. He's discarding all but the final one. It's like a one element buffer.
-
to me your class is a messageBuffer or a bufferedQeue(a buffer it's just a place to store the things generated by the producer until the consumer actually consumes them, or that's what i've been told)
But not all messages are consumed.
I'd go for the word aggregate. As in, you're aggregating any messages that come in in the interval.
Aggregate feels like it should combine multiple things into one. This doesn't do that.
I implemented it as
Debouncer
.
That's a good unique name and it's already used in a popular js library.
-
to me your class is a messageBuffer or a bufferedQeue
Now that I understand what he's doing, it seems like there should be
filter
in the name somewhere.
-
Bottleneck
-
I implemented it as Debouncer.That's a good unique name and it's already used in a popular js library.
It would also confuse the bejeesus out of me - a debouncer, to me, is something that filters out spurious updates, not rate-limits them.
-
I think it's a LIOO stack. It's a whole new paradigm.
-
I like the word "throttled" that people have been using. I would suggest:
ThrottledEventQueue
-
Buy WebTrends, I used to work there, good people.
-
I like the word "throttled" that people have been using. I would suggest: ThrottledEventQueue
To me, queue implies FIFO semantics. That's why I proposed pump - in analogy to WinAPI message pump.
-
Isn't it what he is doing ? He accumulates updates for a while then serves the last. (His initial explanation is inaccurate)
I think it's a LIOO stack. It's a whole new paradigm.
this
-
debounce is within the producer of the messages.
Call it a Bouncer then. Like those guys preventing too many people from entering a club.As an aside, I like the sound of ThrottleGate, regardless of whether it is applicable.
-
-
Is there an official name for this? Like a known design pattern or something?
Flow control?
-
Not sure what to expect.
They have a streaming web analytics product that runs over WebSockets, and is pretty damned slick.
-
LIFORD
-
I like liford.
Vote for liford.
-
Wouldn't it be LILORD then? I assume the last update is the newest one, which is the one he wants to propagate.EDIT: no wait I am dumb. Where is my morning coffee?
-
Next time someone laughs when I say one of the hardest things in programming is naming components correctly, I'll point them to this thread.
-
Next time someone laughs when I say one of the hardest things in programming is naming components correctly, I'll point them to this thread.
Yup. And this is ABOVE AVERAGE as far as name ideas are concerned. A lot of good suggestions.
Any day you don't have to go to thesaurus.com to name your shit is a good day.
-
Any day you don't have to go to thesaurus.com to name your shit is a good day.
When in doubt, just name it Steve. It's a nice name.
-
it seems to behave like a Generic Cell Rate Algorithm with a small bucket. very small. And opposite of usual bucket which keep the first and dismiss the late.
SingleNewestTokenBucket
that, or going to thesaurus for traffic shapping and rate limit