Protocol and infrastructure design of a chat service

helpfulcorn

I've taken on a new hobby of trying to create a sort of chat service, mostly to help me learn to implement network communication in better ways, authentication, etc. The only protocol I really have familiarity with is IRC. Basically what I'm wondering is what would be the best way to store authentication and chat room information on an entire network? At this point in time I can think of:

Having some kind of database each server can connect to, probably caching as well, which stores user information
Go the IRC route and have every server store everything on every user.

Like I said, I'm new to this kind of thing and I'm wondering if anyone has any suggestions as to ways this can be implemented and work well. Unlike IRC though, connection won't be completed until after authentication, so something like IRC services won't really work.

On a side note, the language I'm using is perl.

Any insight, help, links, books, etc would be greatly appreciated!

Thank you

PSWorx

You already seem to have made the infrastructure decision of multiple servers. Why not go the easy route first and have a single server that manages authentication as well as the actual chatting?

helpfulcorn

@PSWorx said:

You already seem to have made the infrastructure decision of multiple servers. Why not go the easy route first and have a single server that manages authentication as well as the actual chatting?

Well, I wanted to also learn a bit more about server-to-server protocols and connectivity and how information should be shared and routed between them.

cHao

Any particular reason you're not starting with IRC? Inventing a whole new protocol wouldn't be too useful, and tubeswise it'd be hard to test -- you'll have a hard time convincing more than just a couple of friends to install or create a client for it, while a bunch of people already have an IRC client. (Scalability can be a huge thing when it comes to application protocols, especially ones that are structured to assume you'll be running multiple servers, and it'd be easier to notice scalability issues on a network that has more than a couple of users.)

PSWorx

@cHao said:

Any particular reason you're not starting with IRC? Inventing a whole new protocol wouldn't be too useful, and tubeswise it'd be hard to test -- you'll have a hard time convincing more than just a couple of friends to install or create a client for it, while a bunch of people already have an IRC client. (Scalability can be a huge thing when it comes to application protocols, especially ones that are structured to assume you'll be running multiple servers, and it'd be easier to notice scalability issues on a network that has more than a couple of users.)

Well, if it's just supposed to be a hobby/learning project, I doubt he is planning to develop the new IRC killer. A few friends might be enough. Besides, you can do a lot of testing already on localhost or using VMs. Last but not least, there are quite a few things that IRC just sucks for. It's not exactly the most simple thing to implement (or at least not as simple as it could be) and most importantly, there is no standard at all for authentication. IRC+ tried but didn't seem to have any success so far.

I agree, though, it's important to test the whole thing with a large number of clients. Even if most of the clients are just bots.

So, ignoring all the conceptual questions, from a purely technical standpoint, I'd start with a simple client/server model first. Later (if you still have time and motivation then) you might try to extend it to multiple servers that each take a number of clients and connect with each other and probably a separate auth server that all the other servers connect to to get user information.(Note: Obviously I'm taking about server applications here, not necessarily separate physical machines)

But the first thing you probably want to do is to make yourself clear how your model looks and what kind of entities you have to manage in that network (in this case users, channels and maybe servers as well). Then you can think of which entities have what roles on the network, where you want to store them and how you want to transmit them over the network.

Have you considered implementing a Jabber server?

zzo38

@helpfulcorn said:

Unlike IRC though, connection won't be completed until after authentication, so something like IRC services won't really work.

You can still use the PASS command in IRC, and have it put a error message in case of wrong or no password given. You could use the combination of PASS/USER/NICK (in that order) for authentication. The client might display a login dialog box, or do it manually, or in a different way. (In my client, I type in PASS and the password in manually, and then use the F1 macro to login. If the command starts with "PASS" it will echo the password as asterisks)

@helpfulcorn said:

1) Having some kind of database each server can connect to, probably caching as well, which stores user information
2) Go the IRC route and have every server store everything on every user.

Even with IRC, it can be programmed differently. If the protocol for server/client is same, the server/server can have a few different commands as long as all servers on that network comply.

For the room/channel, you can do like IRC has different prefix depending on the mode of the channel, # for all of network with modes, & for locally server with modes, + for all of network with modeless, and you can also have ! and . as well.

dstozek

As tster mentioned before, take a deep look into Jabber and XMPP protocol

This is why you might like it:

relatively fresh standard, quite flexible
many different solutions/libraries/plugins already exist for XMPP (you can use them to omit the networking/protocol details and focus on features based on this, although this might not be what you want)
you can write a client first and test it against existing server software (or write a server and test it with existing clients)
your solution will be compatible with existing networks
you can write your own services for new features you might want to have