So, here's a question
-
So we all know about the preview being out of sync with the actual text displayed.
Why the hell doesn't Discourse just pass the raw post back and forth? It's not like whatever vodoo they're doing on the server actually makes it more secure - any of their escaping and what-not should be done with the preview too...
If we're going with the whole "app built in JavaScript" thing, why did they decide not to do that too?
Discuss...
TL;DR: Baking posts is stupid and you should feel stupid
-
Why the hell doesn't Discourse just pass the raw post back and forth?
The most obvious reasons:- Speed (stop laughing at the back)
- Less network traffic
-
But you're sending more anyways - the raw and the cooked are passed both on post and on display. Why do we even need the cooked post to go over the wire at all?
-
I don't think the cooked is passed on post. it is passed on display for quoting purposes, but I don't think it's passed on post, just the raw.
-
It is. At least if I didn't misread the network tab it is...
-
That could be from our </> view raw feature.
-
Granted that the speed argument makes sense. Technically the JS would have to render it. But I'd say the lower network use (about 50% less per post) would probably offset that nicely...
Also you've got the added benefit of HEY LOOK YOUR PREVIEW ISN'T DISCOURSISTANT!!!!!
That could be from our </> view raw feature.
Other way. When I submit a post, the post:cooked param has my post in it...
-
Granted that the speed argument makes sense.
It would make sense if the result was the same from local vs server. But instead the result isn't, so it looks like the product is broken.
-
Yeah pretty much that. I doubt the .0005ms you'd save on rendering speed makes up for that
-
You want to expose all our secret escaping mechanisms?
Oh wait, the raw would be escaped.
They're passing the rendered result back?
-
Yup.
-
They're passing the rendered result back?
Nah. Raw gets sent back, then it gets cooked/baked/whatever IIRC.
It's just the oven on the server and the oven on the client are different.
-
...
Well that means the final render of the post could be different than the rendering in preview, depending on if there's any differences in the two renderings (local vs. server).
-
wait can't repro? wtf?
-
``` Precisely ```.
-
What the fuck?
-
Shit, did you see that too?
-
My understanding is that they behave like this:
new post/edit:
Client (displays raw, cooked client side) -[raw & cooked]-> server (discards cooked presumably, then does its own cooking, then passes back to client to show server-cooked)View post:
Server -[raw & cooked]-> client (displays cooked)
-
yep, and see what?
-
It first posts the rendering from the server, then your client re-renders the raw?
So temporarily you see what you see in preview.... then it changes to client rendering.
Yes.
-
If you mean the post-post post being strikethrough and then not anymore? Yes.
-
Yep!
-
It's the other way around. It client renders (only for you), then sends to server, then re-renders it based on the server's cooking
-
Other way. When I submit a post, the post:cooked param has my post in it...
From desktop, yes; mobile might be raw-only, to save on data usage. And the bots only send raw too.
-
Ah, well at least that makes sense. Still doesn't explain why they don't just PASS THE RAW BACK AND USE THE SAME RENDER ENGINE AS THE PREVIEW...
</discorant>
-
If I edit my post up there with
, it renders one way temporarily, the flashes to another way.precisely
You can figure out which bake is which, but I get both bakes.
-
what? I don't even? How does that happen?
-
-
Yeah
-
How does that happen?
When I edit this post. (Post I'm replying to)
First it appears the same as Preview (with the strikethrough), then a second later it doesn't.
So first it displays the preview cooked, then it displays the server cooked after the post is krunched by the server and returned.
-
Well I know how that happens, I'm just wondering what f**kery makes that actually a valid behavior...
-
Well I know how that happens
sorry, butHow does that happen?
I think it's so your edits and posts show up just that little bit faster.
It's the same thing as when a post is rejected by the server, it first shows up in the thread, then disappears with the popup.
-
Yeah. Sorry, I forgot the
<facetious_statement>
tags
-
Well I know how that happens, I'm just wondering what f**kery makes that actually a valid behavior...
Duckhorse.
Ok, seriously now: I understand why they do it. The only way to run the same parser serverside is using a headless browser or Node. We already have enough fucking services, so them not doing that is fine by me.
The bit I don't understand is how the hell they can't find two compatible parsers. Or fix one of them. From some testing here, all sanitization is already done serverside. That would mean they have to fix up the JS side. Maybe if they stopped all the Discoshedding they could actually do it.
But apparently that's an edge case, so they don't bother. Or they don't want to maintain their own fork of the parser. That's a reason I went for a "lesser" option in some cases myself.
But this is one of the core features for them, at least considering how much Jeff sings praises about Markdown. It makes it "easy" to write posts. I agree. It would be. If both parsers weren't horribly broken.
I can format the post really easily using DiscoMarkBBTML. By which I mean the markup is easy to use, is not overly verbose, and I can picture the results easily. But in the end, all that saved time gets wasted because I have to keep adding newlines, being careful about spacing and escaping shit manually.
This shit should be a priority for them. Not what fucking colour the new post indicators are.
-
Well my point is why do they need to even parse it serverside? It's not like there's a valid reason to... at least from what I can see
-
If a bug gets fixed in the parser they can run a job on the server to re-render all the posts. Because rendering, I'm sorry, "baking" is expensive, apparently, so it's not done on the fly.
And even with all the bugs the serverside one is much, much better at sanitizing crap in it's current state, from what saw at least. Hint from a recent experiment: try nesting polls and look at the preview. Then post that poll. Yeah. That.
Edit: Oh, you mean use raw for everything? Sorry. Well.. one, the JS one is crap, apparently. And I'm pretty sure they want to present HTML to search engine spiders, not raw.
-
Well my point is why do they need to even parse it serverside?
I'm sure I covered that already:
@RaceProUK said:From desktop, yes; mobile might be raw-only, to save on data usage. And the bots only send raw too.
-
Did you?
-
I'm not quoting myself a second timeβ¦
-
That doesn't address the question though... I'm asking why we even need to do any work server-side at all... why can't we just pass the raw when you post, then pass it to the client when the post is requested? In other words, completely remove the concept of server-side baking
-
And even with all the bugs the serverside one is much, much better at sanitizing crap in it's current state, from what saw at least. Hint from a recent experiment: try nesting polls and look at the preview. Then post that poll. Yeah. That.
Edit: Oh, you mean use raw for everything? Sorry. Well.. one, the JS one is crap, apparently. And I'm pretty sure they want to present HTML to search engine spiders, not raw.
Well, they should improve the client one then
Having an issue with the preview window is just as bad IMO.Granted the search engine one is a valid point.
-
You mean, rebake the post client-side every time it's requested? Sure, why not? Run a mobile phone battery flat in 30 minutes, but sure!
-
I refer you back to search engine spiders.
Also, do you really want to let your phone render the post content as well?
'd!
-
Well my point is why do they need to even parse it serverside? It's not like there's a valid reason to... at least from what I can see
You mean besides that it's user input and it should never be trusted?
-
You mean besides that it's user input and it should never be trusted?
Obviously you treat it as such. You should still do all the security things you would do normally... I don't see how this would affect that...
@RaceProUK said:You mean, rebake the post client-side every time it's requested? Sure, why not? Run a mobile phone battery flat in 30 minutes, but sure!
I... don't think it would be that big of an issue... Most webpages run a fair amount of JS anyways. I do see your point though. FWIW, Discourse kills my phone's battery anyways so.... mehI refer you back to search engine spiders.
Well, how exactly would this affect them? The text is still there, just not HTML-ified... IANASEE so I'm legitimately asking :)
-
The only way to run the same parser serverside is using a headless browser or Node
Also, they actually do do that. How they end up with different content, I have NO F**KING clue
That's kinda my point though - they use the same thing (they're doing something, I think it's node, launching javascript to run
handlebars.js
I think...) both server and client side, but they're doing different things. Clearly we shouldn't trust only client input, but we also shouldn't have inconsistency like that...Filed under: Would post a link to Github, but it's Friday night and I really, really CBA
-
@rad131304 said:
You mean besides that it's user input and it should never be trusted?
Obviously you treat it as such. You should still do all the security things you would do normally... I don't see how this would affect that...Well if you don't rebake at the server and just do your security stuff to baked, then the user could submit completely different data for baked and raw.
-
Well if you don't rebake at the server and just do your security stuff to baked, then the user could submit completely different data for baked and raw.
That's not what I'm saying
I'm saying you should bake client-side when you recieve the post, rather than rely on the (obviously buggy) server implementation of the same code
-
@rad131304 said:
Well if you don't rebake at the server and just do your security stuff to baked, then the user could submit completely different data for baked and raw.
That's not what I'm saying
I'm saying you should bake client-side when you recieve the post, rather than rely on the (obviously buggy) server implementation of the same codeI guess that makes less sense to me; why do all of that work over and over again at the client every time the content is served? How do you run the post through an HTML whitelist if you don't bake it first?
I think their architecture strikes the right balance between security, data transmission, and performance - they just need to do a better job of testing.
-
@sloosecannon said:
@rad131304 said:
Well if you don't rebake at the server and just do your security stuff to baked, then the user could submit completely different data for baked and raw.
That's not what I'm saying
I'm saying you should bake client-side when you recieve the post, rather than rely on the (obviously buggy) server implementation of the same codeI guess that makes less sense to me; why do all of that work over and over again at the client every time the content is served? How do you run the post through an HTML whitelist if you don't bake it first?
I think their architecture strikes the right balance between security, data transmission, and performance - they just need to do a better job of testing.
No that's what you're not getting - the baking is the Javascript, and it runs on both the client (preview window) and the server anyways. My point is that for DRY and consistency, in addition to bandwidth performance, it should be rendered client-side.
I can see reasons for the way they're doing it that way too, it's just it reeks of DRY violations and just makes me cringe