Help me name a data structurish thing
-
I need a quick naming tip.
I am making a class like this:
/** * Timed Buffer will retain objects push-ed into it, up to either the "limit" or "delay" (ms), whichever comes first. * It will then call the provided fn with whatever data was pushed during the "buffering" phase. * @param limit How many objects can we store before "releasing" * @param delay After the first object is pushed, how long can we delay between pushes before releasing (debouncing) * @param fn Function to call to "release" * @constructor */ function TimedBuffer(limit, delay, fn) { //TODO } const timedBuffer = new TimedBuffer(3, 1000, (buffered) => { console.log(buffered); // [1,2,3] }); timedBuffer.push(1); timedBuffer.push(2); timedBuffer.push(3); timedBuffer.push(4);
Basically, some events that are "dripping in" are causing costly recalculations. I want to buffer them up and occasionally release them all at once, to try to improve the perf.
So how do I call this data structure which I temporarily named "TimedBuffer"?
This seems like a common enough case, there's got to be some well accepted name everyone else uses. Probably something like spigot or faucet or sieve or some crap like that.
Halp!
-
@cartman82 Looks like a variant of the Burst Buffer?
-
@cartman82 Possible inspiration: C#'s "object cache" class:
Possibly the CacheItemPolicy is helpful:
This called it a "CacheEntryUpdateCallback"
I dunno, maybe not useful. Just a C# class that does similar stuffs.
-
@rhywden said in Help me name a data structurish thing:
@cartman82 Looks like a variant of the Burst Buffer?
Oh cool!
Sounds exactly right!
-
Kinda, sorta? Not really. To be honest the "timed buffer" idea looks perfectly cromulent to me.
-
@cartman82 Yeah, well, it's just that it's already named for a different concept, that's why I said a "variant" thereof.
Just to be clear, that name is what I first thought of and then googled the term to see if it already existed. ;)
Of course, you could always go for "Timed Burst Buffer", patent the idea and call it a day.
-
@blakeyrat said in Help me name a data structurish thing:
I dunno, maybe not useful. Just a C# class that does similar stuffs.
As far as I can tell, this cache callback will be called for each entry individually. I need one callback for a group of items.
Also, Microsoft is NOT the best example of naming things. :)
-
@cartman82 Eh. C#'s designed for maintainability, so their naming is pretty good I think. Verbose, but good.
But yeah, sorry, that class isn't a very close fit. If you were implementing this in C#, you could certainly use it though.
-
@rhywden said in Help me name a data structurish thing:
@cartman82 Yeah, well, it's just that it's already named for a different concept, that's why I said a "variant" thereof.
Yeah, they don't seem to have a timed component, but are using it for similar-ish purpose.
-
BTW I glanced at your top post again and "capacity" would be a better term than "limit", I think.
-
@cartman82
Low-pass filter
Capacitor
-
@adynathos said in Help me name a data structurish thing:
Low-pass filter
Flux Capacitor
-
@adynathos said in Help me name a data structurish thing:
Low-pass filter
I'm failing to see why it'd be a low pass filter. It's not filtering anything AFAIU.
Capacitor would be a good name.@cartman82 out of curiosity. For what are you using this? Is it in a redux selector?
-
@cartman82 said in Help me name a data structurish thing:
@rhywden said in Help me name a data structurish thing:
@cartman82 Looks like a variant of the Burst Buffer?
Oh cool!
Sounds exactly right!
How many people who are going to read this code later know what a burst buffer is? Common terminology is only helpful if it's actually common.
To me, just leaving the "timed buffer" name is fine. It's a buffer. It's dependent on time. It's a nice name that sort-of indicates what the thing is, even if you've invented it yourself.
(OTOH, not checking Wikipedia, a "burst buffer" sounds to me like something that buffers bursts - i.e. a buffer that smooths out the flow by, say, buffering an N-packet burst and outputing each packet every M ms. So the opposite of what yours does.)
-
@cartman82 TldrBuffer
ToBufferOrNotToBuffer
DeBuffer
-
MergeBuffer, PackingBuffer, PacketBuffer?
-
NaglingBuffer after the TCP's Nagle algorithm which seems to kinda match what you're doing.
-
WTB
-
Flyball governor
...would be the opposite of what you're doing.Bamboo water clock? Shishi-Odoshi?
-
@luhmann said in Help me name a data structurish thing:
WTB
Want To Buy? And what do you want to buy exactly?
-
@cartman82 I like @Rhywden suggestion.
Also: GroupingBuffer, ClusteringBuffer, BufferyBuffer, McBufferyBufferFace
-
You've got a pretty standard producer/consumer setup there, except you're passing the consumer into the queue, rather than passing the queue to the consumer.
-
Pause Each Action Now Until Timed-out Buffer
The PEANUT Buffer
-
@mzh The
Cluster And Retransmit Together Buffer
CART Buffer
-
@pleegwat said in Help me name a data structurish thing:
@luhmann said in Help me name a data structurish thing:
WTB
Want To Buy?
Wide Transport Bus
-
@wharrgarbl said in Help me name a data structurish thing:
@mzh The
Cluster And Retransmit Together Moving All Nodes Buffer
CARTMAN Buffer
-
Cluster And Retransmit Together Merging Ansynchronous Nanoevents v82
-
@mzh said in Help me name a data structurish thing:
Pause Each Action Now Until Timed-out Buffer
The PEANUT Buffer
I ran out of likes.
-
@boomzilla said in Help me name a data structurish thing:
Cluster And Retransmit Together Merging Ansynchronous Nanoevents v82
What you did there....
I see it.
-
How about Aggregating Size- or Delay-based Flush Buffer? Or Aggregate before Signaling Delegate Function Buffer?
-
@asdf Timed Recalculations - When Through Flush Buffer
-
What I'm saying is, the problem is usually split into the following three components:
//the queue const buffer = []; //the consumer setInterval(() => { if (buffer.length > 0) { console.log(buffer); buffer.length = 0; } }, 1000); //the producer buffer.push(1); buffer.push(2); buffer.push(3);
If you're limiting the size of the buffer to prevent the producer from getting too far ahead of the consumer, that's called a bounded queue, so maybe an appropriate name for your function would be BoundedQueueConsumer, or something along those lines.
-
@adynathos said in Help me name a data structurish thing:
Low-pass filter
CapacitorLowPassFilter is too fancy and complicated. Capacitor is better, but not quite right.
-
@jarry said in Help me name a data structurish thing:
@cartman82 out of curiosity. For what are you using this? Is it in a redux selector?
Yes.
When my Steam client is executed with a clean slate, it starts loading data from Steam and stuffing it into the store. Each new record triggers regeneration of the game list for the main screen. Since this happens several times a second, it's a problem.
-
@maciejasjmj said in Help me name a data structurish thing:
To me, just leaving the "timed buffer" name is fine. It's a buffer. It's dependent on time. It's a nice name that sort-of indicates what the thing is, even if you've invented it yourself.
Good point. But still, "timed buffer" isn't quite right. It makes it seem like each item will be timed separately, like the class blakeyrat had linked.
And to make it sound right, I'd have to add more awkward words into the name. I am closer to a fancy single word, even if it's not quite as descriptive.
-
@gąska said in Help me name a data structurish thing:
MergeBuffer, PackingBuffer, PacketBuffer?
Maybe PackingBuffer...
-
@robo2 said in Help me name a data structurish thing:
NaglingBuffer after the TCP's Nagle algorithm which seems to kinda match what you're doing.
Not bad, but a naive user might assume the class works exactly as Nagle, in that it expects an ack from a previous callback or something. Might be confusing.
-
@asdf said in Help me name a data structurish thing:
How about Aggregating Size- or Delay-based Flush Buffer? Or Aggregate before Signaling Delegate Function Buffer?
"DelayedFlushBuffer" is a good multi-word name.
-
@buddy said in Help me name a data structurish thing:
If you're limiting the size of the buffer to prevent the producer from getting too far ahead of the consumer, that's called a bounded queue, so maybe an appropriate name for your function would be BoundedQueueConsumer, or something along those lines.
Yes, but I want this class to push out events. Your "consumer" is pulling.
Also, the name "consumer" doesn't really reflect data will be consumed in chunks.
-
I went with the BurstBuffer in the end.
/** * BurstBuffer will retain objects push-ed into it, up to either the "capacity" or "delay" (ms), whichever comes first. * It will then call the provided fn with whatever data was pushed during the "buffering" phase. * @param capacity How many objects can we store before "releasing" * @param delay After the first object is pushed, how long can we delay between pushes before releasing (debouncing) * @param fn Function to call to "release" * @constructor */ export function BurstBuffer(capacity, delay, fn) { let _buffer = []; let _timeout = null; Object.assign(this, /** @lends BurstBuffer.prototype */ { push, release }); function push(item) { clearTimeout(_timeout); _buffer.push(item); if (_buffer.length >= capacity) { release(); } else { _timeout = setTimeout(release, delay); } } function release() { if (_buffer.length) { const out = _buffer; _buffer = []; fn(out); } } }
Thanks, everyone.
-
-
@cartman82 said in Help me name a data structurish thing:
function push(item) { clearTimeout(_timeout); _buffer.push(item); if (_buffer.length >= capacity) { release(); } else { _timeout = setTimeout(release, delay); } }
I'm no JS coder, but don't you have it set up so that pushing an item into the buffer resets the timeout period? Is that intended?
I thought you had wanted it to start the timer when an item was pushed into an empty buffer, and then it would flush the buffer when either that timer expired or the buffer was full. If you have data coming in at a slow rate that is just below the timeout, then you'll be waiting a relatively long time before you get a flush.
-
The WtfCorp name for this is "Holding Tank"
-
@weng I was quite a fan of LeakyBucket (as proposed by @Maciejasjmj) myself.
BurstBuffer seems like a good choice too, if you want to sound all serious.
-
(already solved, I'm aware, no curry)
LeakyBucket is the awesomest.
TimedBuffer is the most boring but clearest, i think that's what I used when I was making basically the same thing
DripBuffer / DripfeedBuffer is my current personal suggestion aiming at explaining it quickly by just creating the right mental image connected to a real thing. you could call it "a skeuomorphic name", if you're a naming junkie like me :-D
-
@djls45 said in Help me name a data structurish thing:
I'm no JS coder, but don't you have it set up so that pushing an item into the buffer resets the timeout period? Is that intended?
Yes. The timeout is for "no further data is coming" not for the buffering time. The duration of the buffering will be limited by the capacity (max capacity * delay).
With your implementation, I'd have to increase the delay to cover the entire potential buffering phase. Eg. if I now have 5 items at 1s delay, I'd have to push the delay to 5s to get the same performance at peak. But then, if I only get one item, I have to wait 5s to see it on the other end, while with my system it's only 1s.
Dunno, it's not that important in the end, I just like it better for the buffering duration to be dependent on the data pressure.
-
@cartman82 It covers a different usecase. Not resetting the timer when another item is added gives you a better guarantee of how long an item will be buffered.
If you are buffering unrelated items that need to go to the same end, that kind of guarantee is probably desirable. Your solution is probably more useful if you are flushing related items, and the first item in the buffer will not be relevant on the other side of your backend processing until the last item has been flushed as well.
-
Maybe "Queue" instead of "Buffer" (e.g., "BurstQueue")? My thinking is that "queue" implies that it will be held for later processing, whereas "buffer" is less well defined (I often encounter the term buffer used for semi-permanent storage with multiple accesses and whatnot).