WTF Bites
-
Oh, updating a single short field on a table causes 30 dead tuples in pg_toast. Found the trigger that causes it, it has many many lines...
Updates not touching the long fields should (in theory) not bloat toast. But I think the definition of not touching is very narrow, eg
SET bloat = bloat || ''
will make dead tuples. What if IDECLARE
and put it back?
-
OK, this is frontpage-worthy.
Imagine a table, let's say
foo(id int primary key, bloat text)
. We insert one record, wherebloat
is larger than 2kB, so it gets toasted.Now, if we update only
id
, the associated toast table doesn't grow, because the record still points to the same data. If we writeSET bloat=bloat
or evenSET bloat = (select bloat from foo limit 1)
, it still doesn't grow. We need to doSET bloat = bloat || ''
to create dead tuples in toast.Now, how else can we create dead tuples? It turns out that all we need is the following trigger:
CREATE OR REPLACE FUNCTION foo_trig() RETURNS TRIGGER LANGUAGE plpgsql volatile AS $$ BEGIN IF NEW <> OLD THEN RETURN NEW; END IF; RETURN NEW; END; $$ ; CREATE TRIGGER foo_trigger BEFORE UPDATE ON foo FOR EACH ROW EXECUTE FUNCTION foo_trig();
The key points are that it must be a
BEFORE UPDATE
(notAFTER
) trigger, and it must containIF OLD <> NEW
. If we reference specific columns, even the toasted ones, it works fine. But once we reference the whole row, it seemingly forgets about the pointers and updates everything.
-
All this talk about bloated toasts is making me hungry.
-
@Tsaukpaetra said in WTF Bites:
All this talk about bloated toasts is making me hungry.
When you put it like that, it's making me not hungry.
-
Terraform has a strange spin on typing. I've already fought with it once and was now forced to remember the hack I got around the issue, so a rant is in order.
Basically, there are two kinds of maps:
map
, which has arbitrary string keys, but all the values are supposed to be of the same type, andobject
, which declares what keys it has, and for each what type is the value.
There are many cases where the type is declared, and checked, but there are also some where you simply build an object to be serialized to JSON and thrown over the fence to some service and that service is much looser with types and may expect varying set of keys depending on other circumstances.
So in this instance I in need of tweaking one such object. In particular, add
metadata.namespace
to a Kubernetes manifest. But only to one in a list. So I wrote something like this:helm_controller_manifests = [ for mf in local.helm_controller_manifests_orig: mf.kind == "Deployment" ? merge(mf, { metadata = merge(mf.metadata, { namespace = kubernetes_namespace.helm_controller.metadata.0.name }) }) : mf ]
First, marvel at the complexity of the expression. Terraform does not have a deep merge (feature request rotting in github issues for years as usual; and it is functional only, so no assignment either), so I have to merge the top level object with an object containing the merged subobject.
But then I get this error:
╷ │ Error: Inconsistent conditional result types │ │ on controllers-helm.tf line 18, in locals: │ 17: mf.kind == "Deployment" │ 18: ? merge(mf, { │ 19: metadata = merge(mf.metadata, { │ 20: namespace = kubernetes_namespace.helm_controller.metadata.0.name │ 21: }) }) │ 22: : mf │ ├──────────────── │ │ kubernetes_namespace.helm_controller.metadata[0].name is "helm-controller" │ │ mf.kind is "Deployment" │ │ mf.metadata is object with 2 attributes │ │ The true and false result expressions must have consistent types. Type mismatch for object attribute "metadata": The 'true' value includes object attribute "namespace", which is absent in the 'false' value. ╵
The
?:
simply does not allow adding members in one of the branches. But I need to add a member in one of the branches, because thekubernetes_manifest
resource does not support specifying the namespace (unlike the command-line client), and of the three resources, theDeployment
needs a namespace, but theCustomResourceDefinition
s don't.The solution is, of course, to rewrite the ternary to:
helm_controller_manifest = [ for mf in local.helm_controller_manifest_orig : { true = merge(mf, { metadata = merge(mf.metadata, { namespace = kubernetes_namespace.helm_controller.metadata.0.name }) }) false = mf }[mf.kind == "Deployment"] ]
That does not complain about the different shapes of the values. Why?
TL;DR: Terraform has a ternary operator
condition ? when_true : when_false
, but in practice you have to write it as{ true = when_true, false = when_false }[condition]
. Because 🦫.
-
@sebastian-galczynski said in WTF Bites:
Hooray architecture astronauts!
Indeed, he now works for one of the
FAANGFAGMAN companies ;)Facebook Apple Google Microsoft Amazon Netflix
FAGMANI like it.
-
-
@Tsaukpaetra said in WTF Bites:
@sebastian-galczynski said in WTF Bites:
Edit: We now have Inner JSON inside CSV. Turns out it can't be edited with Excel, because wrong quotes.
Oh yeah, you're supposed to double-double-quote JSON strings, with escaping.
Guess who found out PHP doesn't do that for you with the default CSV functions?
where numbers use a comma instead of decimal point excel saves CSV with semicolons instead of commas and breaks everything else
the only expression language I know that translate function names and makes googling worse
-
variant called JSONC
variant, called JSON5The superset called YAML seems to be winning.
Except it has its own problems for config files (let's see if I can find that link), and I can't stand a language that demands leading spaces and chokes on tabs.
Edit: Found it: https://www.arp242.net/yaml-config.html
Anyone that has had to troubleshoot a few thousand lines of %s is ready to stab the creators in the face with a rusty spoon.
This holds for all known config formats.
Except for Apache configs.
I didn't find any format I like too. Toml has it's own weird stuff
-
It also auto-corrects stuff that it thinks is a date, but isn't:
Apparently 20% of "scientific" papers in genetics have misspelled gene names for that reason.
-
@sebastian-galczynski this is something we have definitely commented on here before. Whether in this topic or elsewhere, to say, but it has definitely come up before.
Even specifically in research docs like that…
-
@sebastian-galczynski said in WTF Bites:
It also auto-corrects stuff that it thinks is a date, but isn't:
It’s almost as if autocorrect isn’t a good idea when you’re not typing sentences of text …
-
@sebastian-galczynski said in WTF Bites:
It also auto-corrects stuff that it thinks is a date, but isn't:
It’s almost as if autocorrect isn’t a good idea
when you’re not typing sentences of text …FTFY
-
@homoBalkanus said in WTF Bites:
Terraform
You mean OpenTofu?
I heard of it, but really, whatever. I am not only not planning on providing it as a service to third parties, I would not use such service either.
They'll also be both affected by the issue above, because the language is going to remain the same.
And, well, I am generally inclined to prefer the properly open-source version, but here I'd be relying on the plugins made by hashicorp anyway.
Anyway, next time I'm building something, I'm going to prefer pulumi. Because it doesn't use a special- configuration language.
-
@sebastian-galczynski Dunno who the guy is, but
We tell you not to use Excel. You counter with a host of reasons why you have to use Excel. None of them are good reasons. I don’t know what else to say.
QFFT.
Filed under: Excel delenda est.
-
Won't someone think of the middle managers?!
-
@sockpuppet7 said in WTF Bites:
variant called JSONC
variant, called JSON5The superset called YAML seems to be winning.
Except it has its own problems for config files (let's see if I can find that link), and I can't stand a language that demands leading spaces and chokes on tabs.
Edit: Found it: https://www.arp242.net/yaml-config.html
Anyone that has had to troubleshoot a few thousand lines of %s is ready to stab the creators in the face with a rusty spoon.
This holds for all known config formats.
Except for Apache configs.
I didn't find any format I like too. Toml has it's own weird stuff
Toml is the worst, really. It pretends to be simple until you get to the more complex cases, where it becomes weird.
-
@sebastian-galczynski said in WTF Bites:
It also auto-corrects stuff that it thinks is a date, but isn't:
It’s almost as if autocorrect isn’t a good idea
when you’re not typing sentences of text …FTFY
We have a winner, first post after mine already :)
(When I posted it, I was wondering how long it would take for somebody to either
<del>
the second part or quote my post out of context.)
-
... I must be tripping but I don't think that post is what you think it is....
-
@Tsaukpaetra Yes, that post seems to be outside current context.
-
@Zerosquare said in WTF Bites:
The cuts are largely short-sighted
-
I love that sometimes, NodeBB will clear the received likes notifications upon me visiting a thread, and that sometimes it won’t, and I can’t work out when the rules apply to do what.
-
@Arantor I'm guessing it depends on how many posts back it loads
-
I love that sometimes, NodeBB will clear the received likes notifications upon me visiting a thread, and that sometimes it won’t, and I can’t work out when the rules apply to do what.
It seems to me that the rule is that if there are new posts, it clears the likes notifications along with the new posts notification, but if there are not, it only clears the one notification you clicked.
-
@Bulb the fun part is when you enter the thread not at the latest unread but n pages back where it can still end up clearing the new notification but not necessarily all of the likes notifications.
-
@Bulb the fun part is when you enter the thread not at the latest unread but n pages back where it can still end up clearing the new notification but not necessarily all of the likes notifications.
Sounds like we need to bring back the Likes thread, For Science!
-
-
@HardwareGeek see this thing at the bottom? Indicator of pages, equidistant subdivisions of a topic.
-
@Arantor never heard of 'em.
-
@boomzilla I’m telling you, you’re missing out, big time.
-
@HardwareGeek see this thing at the bottom? Indicator of pages, equidistant subdivisions of a topic.
But how would one know when one has reached the end of a thread if it doesn't keep scrolling in new messages as you go? What if one were to navigate to another thread prematurely on accident and thus miss out on important shitposts?
-
accident and thus miss out
Trust me, this happens irregardless if you have paged or cached.
-
I'm getting reports that our embedded JSON-LD contains escape sequences (e.g. \u1234) for Unicode characters.
I can't conceive of why this could be a problem, as that's exactly how Unicode characters are supposed to be encoded. Yet it's breaking something, that I'm sure is a much bigger as any compliant JSON parser should decode these just fine. I'm sure they're using a mess of regex or something instead.
But the encoding is happening automatically as part of our JSON library output, which means the "fix" is going to be some ugly kludge of decoding the escape sequences on our side.
-
@error My first response would be to quote chapter and verse at them. But maybe your encoder has an option somewhere to pick how aggressively it encodes uncommon characters?
-
Toml is the worst, really.
My only reason for not wanting anybody to invent more of those formats is that they would inevitably come up with something that combines the worst features of all of the existing ones (and maybe invents a few new ones).
This space is otherwise a solid example of where a (N+1)th standard would make sense.
-
@HardwareGeek see this thing at the bottom? Indicator of pages, equidistant subdivisions of a topic.
But how would one know when one has reached the end of a thread if it doesn't keep scrolling in new messages as you go? What if one were to navigate to another thread prematurely on accident and thus miss out on important shitposts?
When the page indicator, in blue, be the last box be reached, thou hast reachéd thy Holy Hand Gren— last past.
New posts do stream in while you’re there, so you won’t even miss anything.
-
New posts do stream in while you’re there, so you won’t even miss anything.
Unless the most recent post coincides with a page boundary, in which case you'll just sit and wonder why no posts are being made.
-
@error really? In my experience it just keeps appending posts to the last page disregarding page size limit.
-
@error have not observed, but usually when I hit the end of the thread for now it's back to the unread list because I binge this site like a druggie in need of a fix.
-
reachéd
Who are you, and what happened to the real @Arantor? A true gentleman would rather die than use a French keyboard.
-
@Zerosquare But that can be typed rather easily on a keyboard too: é by ´e.
-
I binge this site like a druggie in need of a fix.
Sadly I can no longer partake of the same. My hardware is not matching the pace of growing software cruft needs, and I think it shows...
-
@Zerosquare said in WTF Bites:
reachéd
Who are you, and what happened to the real @Arantor? A true gentleman would rather die than use a French keyboard.
And iPads conveniently have the ability to type accented letters with a long press.
Though I am currently on a laptop, where my Ctrl and Alt kéys dó wórk fór ácúté áccénts.
-
@Arantor whereas Polish laptops come with an Ąlt key.
-
@Arantor whereas Polish laptops come with an Ąlt key.
I thought they came with a kurwa key?
-
@Arantor whereas Polish laptops come with an Ąlt key.
They also have the dreaded ISO keyboards which I always need to replace with normal (ANSI) ones. And some manufacturers won't sell you an American keyboard in the EU. I smell some "regulatory superpower" in this.
-
@BernieTheBernie said in WTF Bites:
@Zerosquare But that can be typed rather easily on a keyboard too: é by ´e.
It can also be typed on any keyboard with XCompose or WinCompose enabled, usually with ◆e'.
-
@sebastian-galczynski said in WTF Bites:
And some manufacturers won't sell you an American keyboard in the EU. I smell some "regulatory superpower" in this.
Most Czech keyboards have full US-Intl layout marked with the Czech layout added, so not an issue over here.
-
@BernieTheBernie said in WTF Bites:
@Zerosquare But that can be typed rather easily on a keyboard too: é by ´e.
That works on USintl-deadkeys too, although I don't know how anyone, at least of those who occasionally use a command line and didn't learn typing on a 1960s mechanical typewriter, puts up with having to type single quotes as 'SPACE.
-
@sebastian-galczynski said in WTF Bites:
@Arantor whereas Polish laptops come with an Ąlt key.
They also have the dreaded ISO keyboards which I always need to replace with normal (ANSI) ones. And some manufacturers won't sell you an American keyboard in the EU. I smell some "regulatory superpower" in this.
superpower more like. Fewer variants to stock. Our HP and Lenovo suppliers always had them.