Editing posts
-
@anotherusername said in Editing posts:
ah, hell with it and just had the back-end send everything because it's really not that much data anyway.
And that's how a 40k post topic brings a server to its knees, because "not that much data" multiplied over several dozen people trying to interact with the topic at the same time winds up being a lot of fukkin data.
-
@izzion In practice, is it that much different than if all those users were reading/posting in different topics?
-
@hungrier
Yes. In the current case, 90%+ of the data you're sending is unnecessary and wasted. In the hypothetical case, all of the data you're sending is requested and necessary.It's like if every time a McDonald's customer orders a Cheesburger, the kitchen makes one of all of the following:
- Cheeseburger the normal way (Ketchup, Mustard, Onions, Pickles, Cheese, Burger, Bun)
- Cheeseburger with no Ketchup
- Cheeseburger with no Mustard
- Cheeseburger with no Onions
- Cheeseburger with no Pickles
- Cheeseburger with no Ketchup or Mustard
- Cheeseburger with no Onions or Pickles
- Cheeseburger with only Ketchup
- Cheeseburger with only Mustard
- Cheeseburger with only Onions
- Cheeseburger with only Pickles
- Cheeseburger plain
One of those is bound to be what the customer wanted, so what's the difference compared to making just the Cheeseburger the customer ordered?
-
@izzion said in Editing posts:
@anotherusername said in Editing posts:
ah, hell with it and just had the back-end send everything because it's really not that much data anyway.
And that's how a 40k post topic brings a server to its knees, because "not that much data" multiplied over several dozen people trying to interact with the topic at the same time winds up being a lot of fukkin data.
Yeah, and also the day someone decides to post War and Peace in a single post and that gets immediately streamed to every connected user even if they don't care about it...
(I know, it would end up being sent to the users when they want to see the post anyway but 1) that's not all users at the same time so it's likely less stress for the server and 2) from the client's perspective, it's not the same to have the UI wait for some data that is immediately relevant vs. have it block everything else because it's receiving data that'll be discarded with ever being shown to the user)
-
@izzion But if it sent everything out to everyone regardless of what they were looking at, it wouldn't make a difference if 50 people were all posting to the same topic, versus 50 people posting anywhere, since they'd all get it no matter what. In fact in the more efficient case where the server streamed only what was relevant, everyone posting in the same topic at once would be just as bad since they would all be looking at that one topic.
-
@remi Although, to be fair, War and Peace is probably small compared to the tons of useless javascript and other cruft that is sent for a typical web page (I don't remember where was this article measuring page weight in units of War and Peace?), so yeah, it might still end up being fairly small overall...
-
@hungrier
At which point, yes, you would need to optimize that further to compress what you're streaming or something.But that doesn't mean you shouldn't optimize for only sending relevant data first, ala
@masonwheeler said in Editing posts:
@accalia said in Editing posts:
sure? but then you cound't hack the client to do all sorts of awesome things on new posts, and you couldn't do what sockbot does either.
Why not? The workflow would go like this:
Server: New post in topic ID 365, category 4. Post id: 24599
Client: [I don't care about that. Do nothing.]
Server: New post in topic ID 42, category 7. Post id: 24600
Client: [I don't care about that. Do nothing.]
Server: New post in topic ID 555, category 4. Post id: 24601
Client: [I care about that one!] Server, plz send me tehcodezmarkdownz for post ID 24601.
Server:(post content here)
Is there any reason why a scheme like that would cause problems for Sockbot?
-
@izzion Right, I get that spamming everything to everyone is inefficient. But what I was getting at is that
@izzion said in Editing posts:
a 40k post topic
with
@izzion said in Editing posts:
several dozen people trying to interact with the topic at the same time
wouldn't
@izzion said in Editing posts:
bring
sa server to its kneesany more than
@hungrier said in Editing posts:
if all those users were reading/posting in different topics
-
This thread is about deleting posts, and new threads are free 🐀
-
@izzion if your forum was trying to juggle 40k posts per second, you'd probably have some problems... ultimately, socket communications aren't that heavy; it's actually probably more of a big deal that the server's keeping a socket open for each user -- sockets are a finite resource. The amount of actual data isn't that great, particularly since the connection's already open, so there's less overhead than, say, the client making an AJAX request to get the same data.
-
@hungrier
Discourse said "ah, hell with it" and just had the back-end send the entire list of post IDs in a thread (rather than the current window), and now they have an ever-present goal on their Next Release to make it so their forum actually handles 40k post topics with a reasonable performance profile.(I seem to recall they also send the full list of likes on each post too, but I can't be arsed to find proof for that assertion, so it might be a total ass pull)
The point I was trying to make was, 5Kb here and 5Kb there adds up to 1TB pretty quick, especially if you're multiplying that over dozens (or hundreds) of connected users.
-
The more we discuss here, the more I'm tempted to write a simple PoC for a web forum with Infiniscroll and live streaming to figure out whether it's possible to build something better with reasonable effort.
-
@asdf I've had a similar idea (although possibly with less infini-scroll) kicking around in my head for a while. Like "I think I could do better than this in my spare time"
-
@RaceProUK said in Editing posts:
@Yamikuronue said in Editing posts:
Can it be set to a bajillion seconds?
While a good choice, I think a real number would be better :P
Eleventy bajillion should be enough for anyone.
-
@Yamikuronue said in Editing posts:
@anotherusername This was brought up because of things like index posts where you really want to keep adding to it as time goes on. Right now you have to bug a mod to update it later, or be one.
Make everyone an admin. Problem solved.
-
@hungrier said in Editing posts:
I've had a similar idea (although possibly with less infini-scroll) kicking around in my head for a while. Like "I think I could do better than this in my spare time"
What stack would you choose? Because I'd be tempted to use Elm for the frontend (I've always wanted to try it out in a real-world project), and I'd probably choose Python for the backend, mostly because of SQLAlchemy. A basic PoC would support:
- Categories and threads
- Plaintext posts (possibly with image attachments to figure out how to prevent jellypotato)
- Likes
- Infiniscroll (possibly with a paged view as fallback)
- Live post streaming
-
@asdf I hadn't considered any specific technology, other than probably Angular for the front end since I have some experience with it. I was more thinking about how certain features could be implemented, like preventing jellypotato, replies and in-reply-to, polls, up/down/whatever votes etc.
-
@hungrier said in Editing posts:
up/down/whatever votes
I want sideways-votes! No, I have no idea what they mean, but I want them!
-
@Vault_Dweller The thread is
-
@Polygeekery said in Editing posts:
@Yamikuronue said in Editing posts:
@anotherusername This was brought up because of things like index posts where you really want to keep adding to it as time goes on. Right now you have to bug a mod to update it later, or be one.
Make everyone an admin. Problem solved.
My dad used to be a member of a rather anarchist forum that did that as an experiment.
Within a couple of days someone deleted it
-
@Jaloopa I am surprised that it took that long.
-
@anotherusername it's just impossible to recover when a websocket stream break apparently, or someone would have fixed it in nodebb
-
@groo navigating (ajaxy navigation) seems to bring it back online. I haven't figured out any way of detecting whether the connection's dropped, and it doesn't seem to have a set time limit, as far as I can tell.
-
@groo
Why should the server get to have less fun than the users? It just felt like deleting that connection.
-
@Polygeekery Isn't that what does with trust levels?