RabbitMQ WTFs

BernieTheBernie

In a different thread, I told you that I won't be able to post Kevin code anymore. But be assured: I found a great new source of Kevin grade code!

How did that happen?
Well, having a lot of time to spare currently, I thought about learning some current technology. What about a common messaging system? RabbitMQ is seen as some kind of industry standard for that purpose.

RabbitMQ: One broker to queue them all | RabbitMQ

And while going thru their .Net examples and their docs, I decided to open this new thread where I will post many of my findings of and .
Feel free to add your examples, already while I prepare for the first smelly contents.

BernieTheBernie

Let me give my first examples, just with the first tutorial

RabbitMQ tutorial - "Hello World!" | RabbitMQ

<!--

There is a class called Send. That is a verb, isn't it? Class names should use nouns.
In that class, they call a method on an implentation of IModel which is aptly named channel:

channel.QueueDeclare(queue: "hello", ...

That's perfect Kevin English!
Remember DataEdit? Kevin will tell you that it is "Daten bearbeiten" in German, so "Daten" (="data") comes frist, "bearbeiten" (="edit") comes second. Of course, that's the way English has to work, too!

Later on, there is a class called Receive - again a verb instead of a noun.

What a great frist start!

BernieTheBernie

Tutorial 2 does not offer many new s beyond the types shown for tutorial 1.
There is only a BasicQos function of IModel which gets called to influence the behavior. But what does that strange name stand for?
Stupid abbreviation!
What about Basic Quit Ordinary Smoking?

Luhmann

@BernieTheBernie said in RabbitMQ WTFs:

Ordinary Smoking

switching to vaping, uh?

TheCPUWizard

@BernieTheBernie said in RabbitMQ WTFs:

hat's perfect Kevin English!
Remember DataEdit? Kevin will tell you that it is "Daten bearbeiten" in German, so "Daten" (="data") comes frist, "bearbeiten" (="edit") comes second. Of course, that's the way English has to work, too!

Depends on what you want to happen when you run various (mental or programmatic) analysis and soft/group the results.... Do you want all of the "data" items to come together, or all of the different "edit" items...

Same as why yyyy-mm-dd is a preferred date format even though it is not used regularly in any human (not computer) language (that I know of)

dkf

@BernieTheBernie said in RabbitMQ WTFs:

But what does that strange name stand for?

Basic Quality Of Smoking.

Luhmann

@dkf said in RabbitMQ WTFs:

Basic Quality Of Smoking

aka the Weed Quality Index

BernieTheBernie

@dkf @Luhmann Looks like I ought to smoke / vape some good substances now.

dkf

@BernieTheBernie I could have also gone with Quantity of Stupidity. (Those are measured in femto-Trusses, named after the former UK prime minister. We use ordinarily use femto-Trusses because 1 Truss is entirely too large a unit to be practical.)

nerd4sale

@TheCPUWizard said in RabbitMQ WTFs:

Same as why yyyy-mm-dd is a preferred date format even though it is not used regularly in any human (not computer) language (that I know of)

Japanese does. Sort of.

BernieTheBernie

Tutorial 4 offers a new small beyond the known issues of previous tutorials.

In the Main method of the ReceiveLogsDirect class, they create a couple of IDisposable instances before they decide to check if the arguments sent to the method are valid.
Exit quickly, motherfuckers!

Beyond that, they set Environment.ExitCode followed by a return; instead of declaring the Main method as int and leaving it with a return 1 in that case.

Luhmann

@BernieTheBernie said in RabbitMQ WTFs:

@dkf @Luhmann Looks like I ought to smoke / vape some good substances now.

I bet it will make the Rabbits come out and explain the whole thing and the world will make sense ... until you sober up again.

BernieTheBernie

Tutorial 5: stringify.

Messages sent to a topic exchange can't have an arbitrary routing_key - it must be a list of words, delimited by dots.
...
limit of 255 bytes.

Aha. What if a word contains a dot? Are umlaut allowed? Or 7bit ASCII only? Why can't they just use an array?

Let me assume the worst case. Though I'd prefer Wurst .

dkf

@BernieTheBernie said in RabbitMQ WTFs:

Tutorial 5: stringify.

Messages sent to a topic exchange can't have an arbitrary routing_key - it must be a list of words, delimited by dots.
...
limit of 255 bytes.

Aha. What if a word contains a dot? Are umlaut allowed? Or 7bit ASCII only? Why can't they just use an array?

If it's ASCII, can I put a NUL (0) byte in the name?

BernieTheBernie

Tutorial 6 gets even more strigifyed.
Here, they send a remote request to calculate the 30iest Fibonacci number:

var response = rpcClient.Call("30");

Yes, the number 30 is a string (of course!).

And the Call function does a

public string Call(string message)
{
    var messageBytes = Encoding.UTF8.GetBytes(message);

I.e. it sends bytes.
Why not send the bytes of the int? HighByte LowByte issues?
But then I could also say UTF8 WTF42 issues.

On the other end, the number gets restored with:

var message = Encoding.UTF8.GetString(body);
int n = int.Parse(message);

and the result gets sent as the UTF8 bytes of the stringified result.

Furthermore, there is a CorrelationId. It is a string, of course:

var correlationId = Guid.NewGuid().ToString();
props.CorrelationId = correlationId;

BernieTheBernie

Now that the tutorials are worked through, let me dive into the city sewer, i.e. more : the documentation.

.NET/C# Client API Guide | RabbitMQ

<!--

What about that snippet:

props.DeliveryMode = 2;
props.Expiration = "36000000";

A magic ~~mushroom~~number: 2. Fortunately, just a few lines above, we could learn

delivery mode 2 (persistent)

What about an enum? That's like to C# a concept.
I am astonished that they use an int here instead of a string. Be consistent!

Expiration again is what I expect it to be: a string. But what do those 36 millions mean? Years? Seconds? That would be a lilltle more than a year - a long expiration span for a message. Or milliseconds? That would make 10 hours. Could be ok?
If they only documented that in their docs...

But wouldn't it be much better if only .Net had something like a System.TimeSpan thingy, and their .Net client would handle whatever odd calculations to get it into whatever their Erlang written service expects?

dkf

@BernieTheBernie said in RabbitMQ WTFs:

But wouldn't it be much better if only .Net had something like a System.TimeSpan thingy, and their .Net client would handle whatever odd calculations to get it into whatever their Erlang written service expects?

Makes me wonder what can be put in that field, other than a stringified integer.

BernieTheBernie

@dkf What about a Scarfolk.Dont?

BernieTheBernie

@Luhmann said in RabbitMQ WTFs:

I bet it will make the Rabbits come out

I did. The result was devastating.

dkf

@BernieTheBernie I went looking in the docs to RabbitMQ itself, and found this:

The value of the TTL argument or policy must be a non-negative integer (0 <= n), describing the TTL period in milliseconds. Thus a value of 1000 means that a message added to the queue will live in the queue for 1 second or until it is delivered to a consumer. The argument can be of AMQP 0-9-1 type short-short-int, short-int, long-int, or long-long-int.

So there.

No idea why it is a string in the API of the C# client, but the Java client is just the same. It has the feeling of something put in the spec "for flexibility" that has ended up being a carried-forward piece of stupidity.

dkf

@dkf Ah! Found it. Message expiration is defined by AMQP to be a shortstr attribute (at least in version 0-9-1). The spec doesn't define what the legal values for that string are.

RabbitMQ is just handling that basic weird thing. Spec botch. I've seen a few in my time...

dkf

@dkf said in RabbitMQ WTFs:

@BernieTheBernie said in RabbitMQ WTFs:

Aha. What if a word contains a dot? Are umlaut allowed? Or 7bit ASCII only? Why can't they just use an array?

If it's ASCII, can I put a NUL (0) byte in the name?

I found the answer to that too: no. And it's UTF-8 so a ü is fine if encoded correctly.

BernieTheBernie

@dkf said in RabbitMQ WTFs:

@dkf Ah! Found it. Message expiration is defined by AMQP to be a shortstr attribute (at least in version 0-9-1). The spec doesn't define what the legal values for that string are.

RabbitMQ is just handling that basic weird thing. Spec botch. I've seen a few in my time...

But at least, they could make a C# client in a sane way, i.e. narrowing down the allowed set of input values to reasonable things.

I surely would not complain if they prevented me from doing Stupid Things .

dkf

@BernieTheBernie said in RabbitMQ WTFs:

But at least, they could make a C# client in a sane way

Gribnit

@dkf said in RabbitMQ WTFs:

@BernieTheBernie said in RabbitMQ WTFs:

But what does that strange name stand for?

Basic Quality Of Smoking.

Don't use it, it'll straight cough any backpressure.

Gribnit

@dkf said in RabbitMQ WTFs:

@BernieTheBernie said in RabbitMQ WTFs:

But at least, they could make a C# client in a sane way

IDontSeeWhyNot IMeanItsNotLikeItsFundamentallyBrainDamaged

LaoC

@Luhmann said in RabbitMQ WTFs:

@BernieTheBernie said in RabbitMQ WTFs:

@dkf @Luhmann Looks like I ought to smoke / vape some good substances now.

I bet it will make the Rabbits come out and explain the whole thing and the world will make sense ... until you sober up again.

And if you go using Rabbit
And you know you're going to fall
Tell 'em a crackpipe-smoking hipster
Has given you the call
Go ask Bernie
How he was apalled

Bulb

@BernieTheBernie said in RabbitMQ WTFs:

RabbitMQ is seen as some kind of industry standard for that purpose.

Sorta. Kind of. Except when not.

The problem with message brokers is that each uses a different protocol (RabbitMQ is to a rounding error the only one using AMQP 0.9.1), and each has a different feature set. So depending on what you want to do, you may need a different messaging solution every time.

BernieTheBernie

@BernieTheBernie said in RabbitMQ WTFs:

Yes, the number 30 is a string (of course!).

Why, oh why, did I not see what could go wrong so fucking easily?
Actually, a bug which happened to me 25 years ago with a banking application: I wanted to transfer 123.45 Deutsch Mark, butt the systems used different decimal separators - 12,345 Deutsch Mark were transfered...
So, just use a number with a couple of decimals after the decimal separator, or a number with a thousands separator, and, voilà, enjoy!

Kamil Podlesak

@BernieTheBernie said in RabbitMQ WTFs:

@BernieTheBernie said in RabbitMQ WTFs:

Yes, the number 30 is a string (of course!).

Why, oh why, did I not see what could go wrong so fucking easily?
Actually, a bug which happened to me 25 years ago with a banking application: I wanted to transfer 123.45 Deutsch Mark, butt the systems used different decimal separators - 12,345 Deutsch Mark were transfered...
So, just use a number with a couple of decimals after the decimal separator, or a number with a thousands separator, and, voilà, enjoy!

#FirstWorldProblems

BernieTheBernie

@Kamil-Podlesak Would you prefer Indian Rupees and Indian number format? Like 1,00,00,000 rupees?

Gribnit

@BernieTheBernie said in RabbitMQ WTFs:

@Kamil-Podlesak Would you prefer Indian Rupees and Indian number format? Like 1,00,00,000 rupees?

Well that does mirror the pronunciation. 1 hundred lakh rupees.

The Indian civilisation is what happens if you never go back and fix anything

Gribnit

@BernieTheBernie said in RabbitMQ WTFs:

RabbitMQ is seen as some kind of industry standard for that purpose.

Not really... it's seen as the dying bastion of the prior wave of message servers. Kafka is the current target and even now it is aging out, but no replacer known to me yet.

Rabbit is probably not a good idea unless you have literally no other choice.

Gribnit

@Kamil-Podlesak said in RabbitMQ WTFs:

@BernieTheBernie said in RabbitMQ WTFs:

@BernieTheBernie said in RabbitMQ WTFs:

Yes, the number 30 is a string (of course!).

Why, oh why, did I not see what could go wrong so fucking easily?
Actually, a bug which happened to me 25 years ago with a banking application: I wanted to transfer 123.45 Deutsch Mark, butt the systems used different decimal separators - 12,345 Deutsch Mark were transfered...
So, just use a number with a couple of decimals after the decimal separator, or a number with a thousands separator, and, voilà, enjoy!

#FirstWorldProblems

Disagree - usually the first-world currencies aren't in explosive inflation. I mean, usually, don't let the events of the upcoming October throw you.

Kamil Podlesak

@Gribnit said in RabbitMQ WTFs:

Actually, a bug which happened to me 25 years ago with a banking application: I wanted to transfer 123.45 Deutsch Mark, butt the systems used different decimal separators - 12,345 Deutsch Mark were transfered...

#FirstWorldProblems

Disagree - usually the first-world currencies aren't in explosive inflation.

With explosive inflation, shifting decimal point by accident is not a big deal.

@Gribnit said in RabbitMQ WTFs:

@BernieTheBernie said in RabbitMQ WTFs:

RabbitMQ is seen as some kind of industry standard for that purpose.

Not really... it's seen as the dying bastion of the prior wave of message servers. Kafka is the current target and even now it is aging out, but no replacer known to me yet.

Rabbit is probably not a good idea unless you have literally no other choice.

Rabbit and Kafka are absolutely not in the same category and although I don't really have that much experience with Rabbit specifically, I need to emphasize: Kafka is not a good idea unless you really know why you need it.

Luhmann

@Kamil-Podlesak said in RabbitMQ WTFs:

With explosive ~~inflation~~diarrhea, shiftting by accident is not a big deal.

HardwareGeek

@Kamil-Podlesak said in RabbitMQ WTFs:

I need to emphasize: Kafka is not a good idea unless you really know why you need it.

I read Kafka in a high school literature class. I'm certain it's never a good idea.

BernieTheBernie

@HardwareGeek Were the translations of his works so bad? I enjoyed reading Kafka. He is so realistic, after all!

HardwareGeek

@BernieTheBernie said in RabbitMQ WTFs:

@HardwareGeek Were the translations of his works so bad?

The translations were ok, I guess. The situations in which he placed his characters, not so much.

I enjoyed reading Kafka. He is so realistic, after all!

Gribnit

@HardwareGeek said in RabbitMQ WTFs:

@BernieTheBernie said in RabbitMQ WTFs:

@HardwareGeek Were the translations of his works so bad?

The translations were ok, I guess. The situations in which he placed his characters, not so much.

I enjoyed reading Kafka. He is so realistic, after all!

You've never been mechanically tattooed over your entire skin area?

dkf

@Gribnit said in RabbitMQ WTFs:

You've never been mechanically tattooed over your entire skin area?

I think he might have noticed if he had been.

Gribnit

@dkf said in RabbitMQ WTFs:

@Gribnit said in RabbitMQ WTFs:

You've never been mechanically tattooed over your entire skin area?

I think he might have noticed if he had been.

I think people would notice a lot of things.

boomzilla

@Gribnit said in RabbitMQ WTFs:

@dkf said in RabbitMQ WTFs:

@Gribnit said in RabbitMQ WTFs:

You've never been mechanically tattooed over your entire skin area?

I think he might have noticed if he had been.

I think people would notice a lot of things.

Oh, trust me, they do, they just don't want to talk to you.

Bulb

@Gribnit said in RabbitMQ WTFs:

@BernieTheBernie said in RabbitMQ WTFs:

RabbitMQ is seen as some kind of industry standard for that purpose.

Not really... it's seen as the dying bastion of the prior wave of message servers. Kafka is the current target and even now it is aging out, but no replacer known to me yet.

NATS looks interesting, but I am not sure how much users it already accumulated. Everybody seems to be going with whatever their cloud provider offers instead.

Rabbit is probably not a good idea unless you have literally no other choice.

This project here did a comparison (which, sadly, didn't include NATS, probably because they didn't run across it) maybe two years ago and they concluded that Rabbit is the only option that provides guaranteed delivery and large messages.

@Kamil-Podlesak said in RabbitMQ WTFs:

Rabbit and Kafka are absolutely not in the same category and although I don't really have that much experience with Rabbit specifically, I need to emphasize: Kafka is not a good idea unless you really know why you need it.

Indeed. My understanding is that Kafka only offers the mode where the messages are queued and any number of clients can read them, but if none does, the messages silently time out. But many projects need messages that are guaranteed¹ to be processed and IIRC Kafka does not offer that.

¹ In Rabbit queue, if no client acknowledges handling the message in specified timeout, usually 1 day, the message drops into a ‘dead letter' queue because that clearly indicates some part of the system is defunct and needs to be manually diagnosed.

Gribnit

@Bulb If your readers are overrun, you gain very little redemption by trying again tomorrow-maybe from a DLQ, and wart better off to have added more readers. Between at-most-once and at-least-once delivery, if you don't control all the backends then you can't establish the idempotence you need in order to safely use at-least-once. And one assumes that there is also a persistence system somewhere? And a capacity for retry other than queue-driven retry?

_{⁰ Waiting a day for DLQ entry vs seeing a real-time monitored count mismatch, sounds like the industry standard mentality}

Bulb

@Gribnit Sometimes when my readers are overrun, I am perfectly fine to wait until Monday when the operator comes to work and starts some extra reader to process the backlog from the DLQ, but I'd have huge problem if the messages expired in six hours and got lost.

When you absolutely need to process the data in some deadline, you probably want Kafka—and need operations on standby 24/7 and can justify the expense. But there are many cases where catching up a couple of days later when there is an outage is fine and then it's much cheaper to use Rabbit and just have operations only work regular hours.

Kamil Podlesak

@Bulb said in RabbitMQ WTFs:

@Gribnit Sometimes when my readers are overrun, I am perfectly fine to wait until Monday when the operator comes to work and starts some extra reader to process the backlog from the DLQ, but I'd have huge problem if the messages expired in six hours and got lost.

Well, THB, default is one week and you can configure it to be longer. But then you might very easily run of space...

The real issue (and the main difference to "normal" MQ) is that there is no concept of "processed message". Kafka is basically just a database of messages ordered by time; each reader has its own pointer to last read message and after reading, this pointer is moved. That's it, the message remains there until it expires by TTL... or the whole thing runs out of disk space.

So, when you mention "six hours", is that a real setting in real setup? Someone thought "ok, six hours is surely enough"? Well, that is exactly the reason why Kafka should never be the first choice.

... Kafka—and need operations on standby 24/7 and can justify the expense.

This. The very design means that you might either run out of time or out of disk space and you absolutely have to have proper monitoring and proper defined way how to prevent either situation.

Also, recovering from HW failures is fun (in the Dwarf Fortress sense). But that is probably true for most MQ systems.

Bulb

@Kamil-Podlesak I've never set up Kafka itself, but the Kafkaesque messaging provided by Azure defaults to 1 … which … is a bit confusing whether it means 1 day or 1 hour — the list shows it in days, terraform sets it in days, but the label in portal says it's set in hours.

Bulb

@Kamil-Podlesak said in RabbitMQ WTFs:

Also, recovering from HW failures is fun (in the Dwarf Fortress sense). But that is probably true for most MQ systems.

… a couple of years we were trying to configure distributed RabbitMQ (and MariaDB; because customer was like “we want HA hurr durr”). Well, as long as you just took out one node or the other and connected it back, it recovered fine. But when you took down all of them, it would not start back. At all. All the nodes just waited for ~~Godot~~ sync from the other ones and couldn't agree on which one should go first (MariaDB had similar problem and there was a manual procedure described, which I spent quite a bit of time trying to automate).

LaoC

@Bulb said in RabbitMQ WTFs:

@Kamil-Podlesak said in RabbitMQ WTFs:

Also, recovering from HW failures is fun (in the Dwarf Fortress sense). But that is probably true for most MQ systems.

… a couple of years we were trying to configure distributed RabbitMQ (and MariaDB; because customer was like “we want HA hurr durr”). Well, as long as you just took out one node or the other and connected it back, it recovered fine. But when you took down all of them, it would not start back. At all. All the nodes just waited for ~~Godot~~ sync from the other ones and couldn't agree on which one should go first (MariaDB had similar problem and there was a manual procedure described, which I spent quite a bit of time trying to automate).

MariaDB can be a PITA, especially when none of the nodes is convinced it's the last one alive that you can safely restart from. We have one distributed over three locations and they never really go down together but there have been some hard lockups that might as well have been power failures.