Server dropping connections frequently

dkf

I'm seeing a lot of bouncing of the connection to what.thedailywtf.com today (and no problems to other sites). According to the console, it's a mix of errors:

Chrome console log section

Failed to load resource: net::ERR_CONNECTION_CLOSED
socket.io/?EIO=3&transport=polling&t=MNxeaUs:1 Failed to load resource: net::ERR_CONNECTION_CLOSED
socket.io/?EIO=3&transport=polling&t=MNxeboE:1 Failed to load resource: net::ERR_CONNECTION_CLOSED
socket.io/?EIO=3&transport=polling&t=MNxed5a:1 Failed to load resource: net::ERR_SSL_VERSION_INTERFERENCE
socket.io/?EIO=3&transport=polling&t=MNxeeP5:1 Failed to load resource: the server responded with a status of 500 ()
socket.io/?EIO=3&transport=polling&t=MNxenUL:1 Failed to load resource: the server responded with a status of 500 ()
socket.io/?EIO=3&transport=polling&t=MNxf0Mu:1 Failed to load resource: the server responded with a status of 500 ()
socket.io/?EIO=3&transport=polling&t=MNxf1WU:1 Failed to load resource: net::ERR_CONNECTION_CLOSED
socket.io/?EIO=3&transport=polling&t=MNxf2jA:1 Failed to load resource: the server responded with a status of 500 ()
socket.io/?EIO=3&transport=polling&t=MNxf3_H:1 Failed to load resource: net::ERR_CONNECTION_CLOSED
socket.io/?EIO=3&transport=polling&t=MNxf5Ig:1 Failed to load resource: net::ERR_CONNECTION_CLOSED
socket.io/?EIO=3&transport=polling&t=MNxfIgO&sid=4ka1WzFI3Qnj5qITAAAW:1 Failed to load resource: the server responded with a status of 400 ()
socket.io.js:7 WebSocket connection to 'wss://what.thedailywtf.com/socket.io/?EIO=3&transport=websocket&sid=4ka1WzFI3Qnj5qITAAAW' failed: Error during WebSocket handshake: Unexpected response code: 400
i.doOpen @ socket.io.js:7
i.open @ socket.io.js:7
i.probe @ socket.io.js:6
i.onOpen @ socket.io.js:6
i.onHandshake @ socket.io.js:6
i.onPacket @ socket.io.js:6
(anonymous) @ socket.io.js:6
i.emit @ socket.io.js:6
i.onPacket @ socket.io.js:7
n @ socket.io.js:7
e.decodePayload @ socket.io.js:7
i.onData @ socket.io.js:7
(anonymous) @ socket.io.js:7
i.emit @ socket.io.js:6
o.onData @ socket.io.js:7
o.onLoad @ socket.io.js:7
hasXDR.n.onreadystatechange @ socket.io.js:7
socket.io/?EIO=3&transport=polling&t=MNxfIkP&sid=4ka1WzFI3Qnj5qITAAAW:1 Failed to load resource: the server responded with a status of 400 ()
socket.io.js:7 WebSocket connection to 'wss://what.thedailywtf.com/socket.io/?EIO=3&transport=websocket&sid=CroVoUNMtEsIJqzmAABe' failed: Error in connection establishment: net::ERR_CONNECTION_CLOSED
i.doOpen @ socket.io.js:7
i.open @ socket.io.js:7
i.probe @ socket.io.js:6
i.onOpen @ socket.io.js:6
i.onHandshake @ socket.io.js:6
i.onPacket @ socket.io.js:6
(anonymous) @ socket.io.js:6
i.emit @ socket.io.js:6
i.onPacket @ socket.io.js:7
n @ socket.io.js:7
e.decodePayload @ socket.io.js:7
i.onData @ socket.io.js:7
(anonymous) @ socket.io.js:7
i.emit @ socket.io.js:6
o.onData @ socket.io.js:7
o.onLoad @ socket.io.js:7
hasXDR.n.onreadystatechange @ socket.io.js:7
socket.io/?EIO=3&transport=polling&t=MNxfPB5&sid=CroVoUNMtEsIJqzmAABe:1 Failed to load resource: the server responded with a status of 504 ()
socket.io/?EIO=3&transport=polling&t=MNxfQPI&sid=CroVoUNMtEsIJqzmAABe:1 Failed to load resource: the server responded with a status of 504 ()
socket.io/?EIO=3&transport=polling&t=MNxfJCu&sid=CroVoUNMtEsIJqzmAABe:1 Failed to load resource: the server responded with a status of 400 ()
translator.js:7 Missing key in translation token "global:"
n @ translator.js:7
t @ translator.js:277
t @ translator.js:217
t @ translator.js:543
a @ app.js:647
(anonymous) @ app.js:655
WUSA.png:1 Failed to load resource: the server responded with a status of 403 (Forbidden)
WUSA.png:1 Failed to load resource: the server responded with a status of 403 (Forbidden)
votes.js?v=5ah5qmbjl9e:1 Uncaught TypeError: Cannot read property 'message' of null
    at i.<anonymous> (votes.js?v=5ah5qmbjl9e:1)
    at i.onack (socket.io.js:7)
    at i.onpacket (socket.io.js:7)
    at i.<anonymous> (socket.io.js:7)
    at i.emit (socket.io.js:6)
    at i.ondecoded (socket.io.js:6)
    at s.<anonymous> (socket.io.js:7)
    at s.i.emit (socket.io.js:6)
    at s.add (socket.io.js:6)
    at i.ondata (socket.io.js:6)
(anonymous) @ votes.js?v=5ah5qmbjl9e:1
i.onack @ socket.io.js:7
i.onpacket @ socket.io.js:7
(anonymous) @ socket.io.js:7
i.emit @ socket.io.js:6
i.ondecoded @ socket.io.js:6
(anonymous) @ socket.io.js:7
i.emit @ socket.io.js:6
s.add @ socket.io.js:6
i.ondata @ socket.io.js:6
(anonymous) @ socket.io.js:7
i.emit @ socket.io.js:6
i.onPacket @ socket.io.js:6
(anonymous) @ socket.io.js:6
i.emit @ socket.io.js:6
i.onPacket @ socket.io.js:7
i.onData @ socket.io.js:7
ws.onmessage @ socket.io.js:7
WUSA.png:1 Failed to load resource: the server responded with a status of 403 (Forbidden)

Is this just the hosting provider being unusually shit or is it a problem with a recent update to ? (I don't think it is a Chrome problem or a connection problem, as those show up much more widely and Chrome was restarted fairly recently…) @ben_lubar, anybody?

Atazhaia

Just leaving the tab idling I can try and collect all the status codes!

JBert

Yeah, all these dropouts are making my productivity unusually high.

Sooner or later my colleagues are going to get suspicious of my work on all those other days.

ben_lubar

@dkf said in Server dropping connections frequently:

@ben_lubar, anybody?

Anyone who tries this again gets banned.

pie_flavor

@ben_lubar whoops

Weng

@ben_lubar said in Server dropping connections frequently:

@dkf said in Server dropping connections frequently:

@ben_lubar, anybody?

Anyone who tries this again gets banned.

Blood for the blood god

loopback0

@ben_lubar said in Server dropping connections frequently:

@dkf said in Server dropping connections frequently:

@ben_lubar, anybody?

Anyone who tries this again gets banned.

How does that break the forum?

Zecc

I'm curious. What happened?

I can see what the post was meant to do, but how did it bring the site down?
Was it stuck baking the post or something?

pie_flavor

@loopback0 Something to do with baking those into img tags, I'm guessing, 'cos the final post is ~350,000 characters long.

ben_lubar

@pie_flavor said in Server dropping connections frequently:

@loopback0 Something to do with baking those into img tags, I'm guessing, 'cos the final post is ~350,000 characters long.

That's not it, because @barisu was able to successfully post it here: http://zendemo.nodebb.com:4568/post/13

sockpuppet7

@ben_lubar so I guess it was your html sanitizing plugin

ben_lubar

@sockpuppet7 said in Server dropping connections frequently:

@ben_lubar so I guess it was your html sanitizing plugin

Nope, tried that and it didn't have any problems. Ran faster than the other plugins that modify post rendering as well.

I can't figure out which part is breaking. My local test forum breaks when I paste in the bad post content, but none of the pieces individually have any trouble at all.

ben_lubar

Ok, so it's a ReDoS in iframely: https://gist.github.com/BenLubar/4863ac6732d7f5fcddb61ec155610a70#file-iframely-rdos-js

Tsaukpaetra

@ben_lubar said in Server dropping connections frequently:

Ok, so it's a ReDoS in iframely: https://gist.github.com/BenLubar/4863ac6732d7f5fcddb61ec155610a70#file-iframely-rdos-js

GG iFramely you never un-dissapoint .

loopback0

@ben_lubar said in Server dropping connections frequently:

My local test forum breaks

I thought this was the test forum?

boomzilla

@loopback0 It's the remote test forum.

ben_lubar

@loopback0 said in Server dropping connections frequently:

@ben_lubar said in Server dropping connections frequently:

My local test forum breaks

I thought this was the test forum?

Sorry, the staging test forum, not the production one.

topspin

@ben_lubar said in Server dropping connections frequently:

Ok, so it's a ReDoS in iframely: https://gist.github.com/BenLubar/4863ac6732d7f5fcddb61ec155610a70#file-iframely-rdos-js

I don't understand what this means.

On an unrelated note, did @pie_flavor get a SPANKing for making me listen to those boring Intel guys all morning without a means of distraction?

ben_lubar

@topspin said in Server dropping connections frequently:

@ben_lubar said in Server dropping connections frequently:

Ok, so it's a ReDoS in iframely: https://gist.github.com/BenLubar/4863ac6732d7f5fcddb61ec155610a70#file-iframely-rdos-js

I don't understand what this means.

On an unrelated note, did @pie_flavor get a SPANKing for making me listen to those boring Intel guys all morning without a means of distraction?

Regular expression Denial of Service - ReDoS | OWASP Foundation

Regular expression Denial of Service - ReDoS on the main website for The OWASP Foundation. OWASP is a nonprofit foundation that works to improve the security of software.

pie_flavor

@ben_lubar In case it wasn't clear, for everyone else interested, iFramely is parsing HTML with regex.

ben_lubar

@pie_flavor said in Server dropping connections frequently:

iFramely is parsing HTML with regex that NodeJS implements using a naive algorithm in O(2ⁿ) time.

Gribnit

Can confirm