WTF Bites

dkf

@sebastian-galczynski said in WTF Bites:

You can't just solder some wires to the board? Or is the JTAG disabled because those pins double as something else?

It's blocked by another board sitting next to it. There's over a thousand of these things, all packed in as tightly as possible (limited by cooling). The JTAG connection points are on the boards, but facing up; to use them I'd need to pull the board out of the overall system, which would mean disconnecting it from the network and the inter-board network. (And I've been working from home for a bit now for some reason…) I can't connect a debugger; it's an in-service system. (Double-fun: identifying which board I'm using is very difficult.)

Many of the most interesting faults have been due to inter-board chatter that was not expected from the design, and which naturally don't occur at all either in isolation or in small deployments. My favourite was the boot messages that caused ARP overload in the on-board OS leading to outgoing messages being rejected and dropped, a fault that only ever manifested at scale.

Zerosquare

@dkf said in WTF Bites:

I got better options than that! I can send UDP messages up to 256 bytes long.

Then what are you complaining about? You're living in luxury!

cvi

@dkf said in WTF Bites:

I can send UDP messages up to 256 bytes long.

When error messages get lost on the network ... is that what people refer to as self-correcting?

dcon

@HardwareGeek said in WTF Bites:

Grill brushes are sold by the pound? I guess?

When did Texas move to the UK?

Gribnit

@dkf said in WTF Bites:

@Zerosquare said in WTF Bites:

use one of the output pins to send bit-banged serial.

I got better options than that! I can send UDP messages up to 256 bytes long.

Until you need to debug the UDP implementation

Tsaukpaetra

@dkf said in WTF Bites:

My favourite was the boot messages that caused ARP overload in the on-board OS

Imagine that should get caught on boot except that it's still booting?

dkf

@Zerosquare said in WTF Bites:

@dkf said in WTF Bites:

I got better options than that! I can send UDP messages up to 256 bytes long.

Then what are you complaining about? You're living in luxury!

https://youtu.be/IeXMKygwSco

dkf

@Tsaukpaetra said in WTF Bites:

@dkf said in WTF Bites:

My favourite was the boot messages that caused ARP overload in the on-board OS

Imagine that should get caught on boot except that it's still booting?

When a board is powered on, it has in ROM a small bootloader that listens promiscuously to the network for packets to load into memory and run as the OS. (It's a nasty protocol in other ways too.) The bootloader also broadcasts a message every second or so saying “I'm here and I want to boot” in order to enable auto-discovery. When you have over a thousand of these things, that's a message on average every millisecond (with random intervals between them), and that was often enough to trigger (with low-ish frequency) infelicities in the OS (the thing that we loaded on) that resulted in the ARP cache getting broken, resulting in it being impossible to send outgoing UDP messages. The infelicity in question was an error in hardware lock management — but only on one code path — that left a short window when the network hardware could overwrite a metadata buffer while it was still being read by the OS. The fault doesn't occur in isolation, and virtually never happens in small scale deployments either (the chance of it occurring is very small even when machine management patterns don't totally exclude it). You only see it at scale, and even then it's pretty rare, but catastrophic.

Finding it was awful. It was a lot of staring at OS code to see if “are we doing anything wrong at this point?” and I was very lucky to have spotted the problem so quickly, in only around a day or so. (It was the second-to-last major issue found in our OS code; the last one was to do with synchronization between clock domains and was not something I worked on.)

topspin

Homebrew is a nice package manager in theory. The down-side is that it sometimes feels that everything you install downloads about 1 or 2 Linux distributions worth of dependencies first.

And sometimes it just downloads dependencies:

$ brew install cask
=> Installing dependencies for cask: coreutils, bdw-gc, m4, libtool, guile, libidn2, libtasn1, nettle, p11-kit, libevent, c-ares, jemalloc, libev, nghttp2, unbound, gnutls, jansson and emacs
...
=> Installing cask dependency: emacs
==> Pouring emacs--27.2.arm64_big_sur.bottle.tar.gz
🍺  /opt/homebrew/Cellar/emacs/27.2: 4,012 files, 104.3MB

topspin

And another nice one from pdflatex:

! LaTeX Error: Unknown graphics extension: .0000.png.

Well, maybe because that's not the extension!

Kamil Podlesak

@topspin said in WTF Bites:

Homebrew is a nice package manager in theory. The down-side is that it sometimes feels that everything you install downloads about 1 or 2 Linux distributions worth of dependencies first.

And sometimes it just downloads dependencies:
$ brew install cask
=> Installing dependencies for cask: coreutils, bdw-gc, m4, libtool, guile, libidn2, libtasn1, nettle, p11-kit, libevent, c-ares, jemalloc, libev, nghttp2, unbound, gnutls, jansson and emacs
...
=> Installing cask dependency: emacs
==> Pouring emacs--27.2.arm64_big_sur.bottle.tar.gz
🍺  /opt/homebrew/Cellar/emacs/27.2: 4,012 files, 104.3MB

What's the issue? EMACS is a completely fine OS, its only flaw being the lack of good text editor.

Gribnit

@topspin said in WTF Bites:

And another nice one from pdflatex:
! LaTeX Error: Unknown graphics extension: .0000.png.
Well, maybe because that's not the extension!

Musta been a fucking windows idiot trying to unix.

Bulb

@Gribnit This is TeX. It has a whole bunch of onion on its belt. It remembers when Windows was a kid.

Gribnit

@Bulb yup. And, it's LaTeX and is being ganghumped by college students if not worse.

dkf

@Kamil-Podlesak said in WTF Bites:

What's the issue? EMACS is a completely fine OS

It's probably being used as a LISP scripting language implementation. Because of course it is.

(I use macports instead. The huge dependency graph thing is still an issue, but that's not really the package manager's fault.)

Bulb

@dkf said in WTF Bites:

It's probably being used as a LISP scripting language implementation.

I could believe that if it wasn't also pulling in guile, which is the Scheme (dialect of LISP) implementation.

dkf

@Bulb said in WTF Bites:

@dkf said in WTF Bites:

It's probably being used as a LISP scripting language implementation.

I could believe that if it wasn't also pulling in guile, which is the Scheme (dialect of LISP) implementation.

Bulb

@dkf Because elisp is as butt-with-warts-ugly dialect of LISP as lisp dialects go.

Gribnit

@Bulb said in WTF Bites:

@dkf Because elisp is as butt-with-warts-ugly dialect of LISP as lisp dialects go.

So, it can be used to accomplish real tasks you say?

Bulb

@Gribnit So can COBOL.

Gribnit

@Bulb said in WTF Bites:

@Gribnit So can COBOL.

No

NO

loopback0

Someone linked this elsewhere and while yes, it's Huffpost, the fact there's even an option for "I'm not sure how I pronounce the word GIF" let alone that high a percentage seems a bit

Gribnit

@loopback0 how do you pronounce the word ghirrarfulics?

PleegWat

@loopback0 said in WTF Bites:

Someone linked this elsewhere and while yes, it's Huffpost, the fact there's even an option for "I'm not sure how I pronounce the word GIF" let alone that high a percentage seems a bit

D: With that horrible throat noise north-Dutch call a g.

dcon

@Gribnit said in WTF Bites:

@loopback0 how do you pronounce the word ghirrarfulics?

gesundheit

Tsaukpaetra

@Gribnit said in WTF Bites:

@loopback0 how do you pronounce the word ghirrarfulics?

Gear-are full ee's

Gribnit

@Tsaukpaetra said in WTF Bites:

@Gribnit said in WTF Bites:

@loopback0 how do you pronounce the word ghirrarfulics?

Gear-are full ee's

Thanks! I did not know. Which means no-one ever can, or that you're right.

TimeBandit

How the hell was this delivered to my gmail address

cvi

@TimeBandit Huh? You're showing just parts of the message, nothing there really matters. If you sent an email with the recipient in BCC, they wouldn't see their address anywhere in those fields either. The actual recipient field is sent separately. If you look at the full headers, you can find that.

See https://en.wikipedia.org/wiki/Simple_Mail_Transfer_Protocol#SMTP_transport_example.

(Or maybe I'm misunderstanding what you're asking.)

Benjamin Hall

@cvi Yeah. Those fields in the body are set completely arbitrarily by the sender, and little if any validation is done on them. You can send a message to anyone, pretending to be anyone else, and that the message was sent to anyone at all, all independent of each other. Have to use a non-webmail client though. You can do it though telnet.

TwelveBaud

@Benjamin-Hall Less doable now in the age of Sender Policy Framework, DomainKeys, and DMARC; you have to at least be able to change records in your From domain's DNS zone. In this example, whoever sent this bought a domain and set everything up correctly for that domain, so Gmail wouldn't know any better even though that domain is crap. I have a plugin on my mail server to do an RDAP query on all sender's domains in addition to the usual anti-forgery stuff and block any domain younger than 14 days old, which knocks out almost 100% of the crap that tries to get through.

Now if only I could figure out who was actually sending it and collect my tree fiddy...

LaoC

@dkf said in WTF Bites:

@Tsaukpaetra said in WTF Bites:

@dkf said in WTF Bites:

My favourite was the boot messages that caused ARP overload in the on-board OS

Imagine that should get caught on boot except that it's still booting?

When a board is powered on, it has in ROM a small bootloader that listens promiscuously to the network for packets to load into memory and run as the OS. (It's a nasty protocol in other ways too.)

From the ~~Little~~Big Book of Things that Should Never Ever Be on the Internet: RKMCEAAS

LaoC

@Bulb said in WTF Bites:

@Gribnit This is TeX. It has a whole bunch of onion on its belt. It remembers when Windows was a kid.

Didn't exactly make it less obscure when they were in the habit of encoding a myriad font variants as 8.3 names—which were already an improvement on the original 6.3 TOPS-10 naming scheme.

LaoC

@loopback0 said in WTF Bites:

Someone linked this elsewhere and while yes, it's Huffpost, the fact there's even an option for "I'm not sure how I pronounce the word GIF" let alone that high a percentage seems a bit

The ones who gave a straight answer understood the question as "How do you pronounce the word GIF" while the others thought it meant "How does one correctly pronounce the ~~word~~acronym GIF".
English language is English.

TimeBandit

@cvi said in WTF Bites:

an email with the recipient in BCC

I need more

Tsaukpaetra

Status: Not sure if a WTF Bite, but apparently if I create an file that would be invalid in Windows on my NAS (i.e. with colons in the name because some lovely person smashed the timestamp as-is into it), Windows sees a randomly-generated 8.3 name, courtesy of the NAS I guess.

At least Windows would attempt to make the generated name kinda-somewhat reasonably-similar to the original, but no...

https://cdn.discordapp.com/attachments/707628100728520788/849039972886773760/unknown.png

From (well, not original, imagine the underscores are colons because I've since fixed them):

Atazhaia

@Tsaukpaetra Clearly, your video collection could be improved by some magic to leverage the power of SSDS and long file names.

acrow

@aitap said in WTF Bites:

Bonus WTF:

$ for host in \
> feed43.com danluu.com sandraandwoo.com \
> phdcomics.com www.antipope.org
> do host $host; done | grep 'has address'
feed43.com has address 145.253.3.148
danluu.com has address 145.253.3.148
sandraandwoo.com has address 145.253.3.148
phdcomics.com has address 145.253.3.148
www.antipope.org has address 145.253.3.148
$ whois 145.253.3.148 | grep role:
role:           Vodafone Germany IP Core Backbone

Why does the ISP even do that? Especially for TLS, which requires SNI to work at all.

Because you only have a publicly routable IPv6 address, and those sites are on an IPv4 address.

There are not enough IPv4 addresses to go around. Shenanigans ensue.

Bulb

@acrow It does not matter whether “you” have a publicly routable IPv4 address, nor how many masquerades you are behind. As long as the servers do have theirs, they can be accessed from behind masquerades using those addresses. No, the reason is clearly they actually use the same CDN.

Also “you” have an IPv4 address, otherwise the records would have to be 6-to-4 ones (and there are several different schemes for 6-to-4—we had to rework a protocol that sent additional IP addresses to connect to to use hostnames, because iPhones stopped setting up IPv4 when IPv6 was available and the only way to find the corresponding 6-to-4 address is by asking the DNS resolver in the IPv6-only network).

dkf

@LaoC said in WTF Bites:

@dkf said in WTF Bites:

@Tsaukpaetra said in WTF Bites:

@dkf said in WTF Bites:

My favourite was the boot messages that caused ARP overload in the on-board OS

Imagine that should get caught on boot except that it's still booting?

When a board is powered on, it has in ROM a small bootloader that listens promiscuously to the network for packets to load into memory and run as the OS. (It's a nasty protocol in other ways too.)

From the ~~Little~~Big Book of Things that Should Never Ever Be on the Internet: RKMCEAAS

It's not. It's on our intranet at work, and can only be routed to if you're using our VPN (or one of the services we stand up in front of it). We used to have some on the internet, but the performance for access to them varied between poor and abysmal (depending on the mood of the intervening ISPs) so we dropped doing that even before worrying about security.

Anyone capable of getting cryptomining going on that sort of hardware can just write that up and get a very good paper out of it. And a block from ever using our systems.

BernieTheBernie

@dcon said in WTF Bites:

@Gribnit said in WTF Bites:

@loopback0 how do you pronounce the word ghirrarfulics?

gesundheit

gezondhijd?

acrow

@Bulb said in WTF Bites:

we had to rework a protocol that sent additional IP addresses to connect to to use hostnames, because iPhones stopped setting up IPv4 when IPv6 was available and the only way to find the corresponding 6-to-4 address is by asking the DNS resolver in the IPv6-only network

This implies that your service didn't have IPv6 addresses to send instead. While using hostnames likely fixes the issue for good, it does make me wonder about the progress of IPv6 adoption.

I've steered clear of and ignored the whole issue for quite some time now, since it has relatively little to do with my day job. Hence my perhaps-ignorant remark. I gather I'll need to pick up a book on IPv6 usage one of these days. But not before the "several difference schemes" phase has passed, I hope.

dkf

@acrow said in WTF Bites:

it does make me wonder about the progress of IPv6 adoption

Varies wildly from place to place. Lots of Asia is IPv6 because they ran out of addresses hard some time ago, whereas the US is still strongly on v4 for the most part. Europe is somewhere between.

Bulb

@acrow said in WTF Bites:

@Bulb said in WTF Bites:

we had to rework a protocol that sent additional IP addresses to connect to to use hostnames, because iPhones stopped setting up IPv4 when IPv6 was available and the only way to find the corresponding 6-to-4 address is by asking the DNS resolver in the IPv6-only network

This implies that your service didn't have IPv6 addresses to send instead.

No, it didn't. I am not sure OVH actually offered them at all at that point (it's been six or seven years).

While using hostnames likely fixes the issue for good, it does make me wonder about the progress of IPv6 adoption.

My impression is that almost nobody actually wants it.

The IPv4 addresses are simple, everybody (who does network setups) remembers the private ranges, remembers the important addresses in their network and so on. But IPv6 addresses look like a scattered tea to everybody. Nobody wants to have anything with them.

And with masquerades moving from being ugly hacks to a highly desirable feature few feel the pressure to switch.

See, I've been setting up some virtual networks in Azure recently. It's all private-range IPv4 only. Because everybody wants it to be private ranges, so why bother with IPv6?

I've steered clear of and ignored the whole issue for quite some time now, since it has relatively little to do with my day job. Hence my perhaps-ignorant remark. I gather I'll need to pick up a book on IPv6 usage one of these days. But not before the "several difference schemes" phase has passed, I hope.

IPv4-only is common almost everywhere. Some ISPs are pushing IPv6 for mobiles because trying to squeeze everybody into the 10.0.0.0/8 block is becoming difficult (even if not exhausted, too much fragmentation makes routing difficult), but for most of the rest IPv4 still works well enough to not bother learning and setting up IPv6.

topspin

@LaoC said in WTF Bites:

@loopback0 said in WTF Bites:

Someone linked this elsewhere and while yes, it's Huffpost, the fact there's even an option for "I'm not sure how I pronounce the word GIF" let alone that high a percentage seems a bit

The ones who gave a straight answer understood the question as "How do you pronounce the word GIF" while the others thought it meant "How does one correctly pronounce the ~~word~~acronym GIF".
English language is English.

Parts of the 65+ population surely also thought "WTF is GIF"?

PleegWat

@Bulb said in WTF Bites:

IPv4-only is common almost everywhere. Some ISPs are pushing IPv6 for mobiles because trying to squeeze everybody into the 10.0.0.0/8 block is becoming difficult (even if not exhausted, too much fragmentation makes routing difficult), but for most of the rest IPv4 still works well enough to not bother learning and setting up IPv6.

I suspect the same applies for corporate intranet.

Parody

@Tsaukpaetra said in WTF Bites:

Status: Not sure if a WTF Bite, but apparently if I create an file that would be invalid in Windows on my NAS (i.e. with colons in the name because some lovely person smashed the timestamp as-is into it), Windows sees a randomly-generated 8.3 name, courtesy of the NAS I guess.

This reminds me of a tangentially similar issue from the days of yore. On the Classic MacOS, files were assigned a (per-volume) unique ID number that Mac programs could use to open the file even if it'd been moved, renamed, or otherwise changed since the last time it was needed. On the Mac side our program would use this first if it was present in a file reference.

Enter Windows NT/2000's Services for Macintosh. Our users started reporting a weird bug where our program would sometimes try to open random files from their Windows-based file server when they opened a document on their Mac. As it turns out, Services for Macintosh's file sharing didn't guarantee those unique ID numbers would never change for a given file, even when it hadn't been moved/renamed/changed.

We ended up adding a preference to make the program check that file references on a Windows file server also matched path and filename.

BernieTheBernie

@topspin said in WTF Bites:

@LaoC said in WTF Bites:

@loopback0 said in WTF Bites:

Someone linked this elsewhere and while yes, it's Huffpost, the fact there's even an option for "I'm not sure how I pronounce the word GIF" let alone that high a percentage seems a bit

The ones who gave a straight answer understood the question as "How do you pronounce the word GIF" while the others thought it meant "How does one correctly pronounce the ~~word~~acronym GIF".
English language is English.

Parts of the 65+ population surely also thought "WTF is GIF"?

Rather: "What's WTF?"

cvi

@BernieTheBernie said in WTF Bites:

Rather: "What's The Fuck is WTF?"

FTFY.

Zecc

@cvi bash.org is