Identity Tokens - facebook/google/flickr



  • I have seen software that generate a token.

    The token is generated if user name and password are correct.

    Then the request made is signed with the token.

    Then the token is valid for certain period of time. Based on token, the webservice is able to identify user.



    So how are these tokens generated? Any idea?



  • Clicky for my personal method. Disclaimer: may be stupid, but so far I've explained it to a lot of people and a lot of people haven't found any problems with it.



  • @blakeyrat said:

    Clicky for my personal method. Disclaimer: may be stupid, but so far I've explained it to a lot of people and a lot of people haven't found any problems with it.



    Ok. I am going to attempt to translate that to some English I can understand.

    Say you have a windows server domain adp (call it what you want).

    On this server you have got username: BlakeyRat password:Clicky.

    Now in order to generate the token, I will take username + password and create some kind of hash. To keep stuff simple, if say I use Caesar encryption method and come up with simple token. This token will look like EODNHBUDW FOLFNB Then I add current date time to this. So it will look like EODNHBUDW FOLFNB 17-APR-2014 06:44:36 PM


    Then when I receive this token, on the server I will decrypt it and confirm that it is indeed Blakeyrat who is signing in.
    Is that the general idea of this game?


  • Kind of.

    The token I create consists of this:

    (some padding bytes) + (known delimiter) + username + password + token expiration date + (known delimiter) + (some padding bytes)

    I then take that and encrypt it and then Base64 it to make it web friendly and then pass it back. This differs from yours in a couple ways:

    1) Both username and password are inside the token, this is so any web server can auth the user without having to talk to any other web server. If your app is load balanced between X servers, this saves having to keep every server's sessions in sync.

    2) The expiration date is inside the encrypted portion. This prevents the user from tampering with it (remember: cookies are plain text), and also gives you the ability to make tokens that expire at different times. Every time a server authenticates a token, it will return a new token that is identical except for the expiration date being incremented* (and of course the padding bytes being different.)

    You can also add some data inside the token as long as you don't go crazy-- the token isn't a full session, so don't add a full session's worth, but in my app I added a few bools that saved DB lookups, like whether the user was an admin or not. Do keep in mind that the entire token, including padding, and including Base64, has to fit inside a cookie, so keep it under, say, 1k.

    *) It's actually a bit more complicated than that, since you usually have long-term tokens created for people who had "keep me logged-in" checked on the login page. Normal tokens last an hour, but before you return the token to the user you want to check that you aren't actually *subtracting* time from the token-- if the token's expiration is already more than an hour in the future, you want to return the same token they authenticated with in the first place. Otherwise, you want to return a token whose expiration is current time + 1 hour.

    That way, the "stay logged-in" user won't get an auth request right in the middle of their active session, but once their session expires (after their two-week token expires) they'll have to re-auth for the next session.



  • It occurs to me that it might be more secure if, instead of username and password in the token, you put in the username and password hash. That way in the monumentally unlikely scenario that an attacker breaks your token encryption, they still don't have access to the original password. The code I wrote might already do this, I'd have to look...



  • @pjt33 said:

    Including the password in the token sounds like a bad idea. It also seems unnecessary.

    And you don't say anything about MAC. Are you using an AES mode with built-in MAC?

    You're right, see my new post. It's been years since I implemented this system, so I honestly don't remember if I included the password or the password hash-- but using the hash instead seems like a no-brainer, so do that.

    (Including both user and password/passwordhash is necessary because my load-balanced servers have no shared sessions.)

    I don't know what "MAC" stands for in this context.



  • @blakeyrat said:

    @pjt33 said:
    Including the password in the token sounds like a bad idea. It also seems unnecessary.

    And you don't say anything about MAC. Are you using an AES mode with built-in MAC?

    You're right, see my new post. It's been years since I implemented this system, so I honestly don't remember if I included the password or the password hash-- but using the hash instead seems like a no-brainer, so do that.

    (Including both user and password/passwordhash is necessary because my load-balanced servers have no shared sessions.)

    I don't know what "MAC" stands for in this context.


    Ok. including a HASH instead of the password is a good idea and I am liking this already.

  • Java Dev

    I wouldn't trust authorization data stored in the token - not so much because the token can get hacked, but because this scheme does not allow for invalidating tokens. So if someone loses admin credentials but still has a token for 2 weeks they keep their access. You could do something like showing links but checking the database before providing any actual access, but that feels like a lot of extra error-prone code.



  • Where the fuck were you people like a year and a half ago when I asked advice on this? Sheesh.

    In any case, you can still invalidate the token easily enough-- whenever any of the web servers receives the token, they still have to authenticate the user. (They don't just take it on faith.) So if the bits that come back from the database say, "oh hey this guy's no longer an admin", the page will not only treat them as non-admin but the new token it generates will also be a non-admin token. So I don't see invalidation as an issue; given my app was simple and only had like 3 levels of permissions. It does make it pointless to keep data like "isAdmin" in the token if you're going to have to look it up anyway 100% of the time, but.

    The problem I was solving is that servers don't have any way of sharing sessions among themselves, so keeping authentication in a normal session cookie wasn't an option. What they *do* all have is access to the user database.



  • @blakeyrat said:

    It occurs to me that it might be more secure if, instead of username and password in the token, you put in the username and password hash. That way in the monumentally unlikely scenario that an attacker breaks your token encryption, they still don't have access to the original password. The code I wrote might already do this, I'd have to look...

    If it's monumentally unlikely that an attacker will break the token encryption then there's no need to include either the password or the hash: the existence of the token communicates the correctness of the password without the need for any shared session.

    @blakeyrat said:

    I don't know what "MAC" stands for in this context.

    Message authentication code. A secure hash of the ciphertext to prevent attacks which modify a byte partway through in the hope that the username will survive intact and the expiry time or extra information will be changed in beneficial ways.

    @blakeyrat said:

    Where the fuck were you people like a year and a half ago when I asked advice on this?

    I don't read every thread, and even less when they're long. If I hadn't been pretty bored then I wouldn't have read this one, because any thread Nagesh starts is bound to be an attempted troll.



  • @pjt33 said:

    If it's monumentally unlikely that an attacker will break the token encryption then there's no need to include either the password or the hash: the existence of the token communicates the correctness of the password without the need for any shared session.

    Well fair enough, but now you're arguing for a design that would make retracting tokens impossible and mine isn't that way.

    @pjt33 said:

    Message authentication code. A secure hash of the ciphertext to prevent attacks which modify a byte partway through in the hope that the username will survive intact and the expiry time or extra information will be changed in beneficial ways.

    I dunno; I just use the .net implementation. If that is the correct thing to do, odds are Microsoft is doing it by default.

    The entire point of me posting this design in the first place is so people who know security could critique it. The problem is, now it's like 18-20 months later, I no longer work at the company I wrote the app for, and so what I did is already chiseled in stone.

    But this conversation might help Nagesh anyway at least.



  • @pjt33 said:

    @blakeyrat said:
    Where the fuck were you people like a year and a half ago when I asked advice on this?

    I don't read every thread, and even less when they're long. If I hadn't been pretty bored then I wouldn't have read this one, because any thread Nagesh starts is bound to be an attempted troll.


    I don't troll here. Just on main articles page.

    Also, I am victim of unusual amount of people trying to look cool like me.



    1. You say you're using the token to sign the request? Or are you just including it with the request? Is the token there to authenticate that the request is valid, or is it to actually sign the request with? If you only want to authenticate, don't sign the request with the token. Just have all requests go over SSL (which will handle encrypting and validating the data) and use the token to authenticate.

    2. Do you need the token to be validated statelessly? In other words, can you have the token in your server's database and validate against that for each request? If so, then what you're doing is creating a session token. Almost every web framework out there supports sessions. Just use sessions.

    3. If it needs to be stateless (e.g. the token can only be validated on its own and can't be checked against a database) then you need to include all relevant data and then sign that data.

      3a) What I do for this is use JSON. I put whatever parameters I want into a JSON object (like when the token expires, what user it is for, etc.). When all of my parameters are in there, I sort the JSON object by key name (so that it's in a "canonical" order) and get the string of the JSON object. HMAC the string using a private key that is kept secure on your server. The key needs to be randomly-generated and the same among all servers (or else the servers cannot authenticate each other's tokens.)

      3b) I then take the signature and put it into the JSON object under a key like "signature".

      3c) Get the string of that JSON object. Base64-encode. Now you have a token. Users can pass it back with the request and your server then validates it by:

      3d) Base64-decode the token, which gives you a JSON string. JSON-decode that. Be sure to check for errors after each of these because the Base64 string or the JSON might be invalid.

      3e) Copy the JSON object, but remove the signature key. Store the signature in a separate variable. Sort this JSON object by key names again (so they're in the same order as when you JSON-encoded the object). Convert to a string and check the HMAC against the signature variable. They should match. If not, the token is invalid.

      3f) If you've made it this far, you have a JSON object with keys in it that you know hasn't been tampered with (unless someone got your private key, which is why you keep it very secure.)

      3g) Now you can check things like "Is the 'expires' key in the past? If so, the request is invalid and the user needs a new token."

      3h) If you store something like a user ID in the JSON object then you can look up the user and check his permissions. This lets you invalidate a user or change his permissions and have that reflected in any tokens. However, if you're validating against a database then you should be using a session in the first place.

      3i) Assuming you aren't validating against a database, then you need to include any relevant permissions in the JSON object. Downside: even if you disable a user, change his permissions or what-have-you his tokens will still be valid until they expire. However, you won't need a database to validate these tokens, so there's that.

    Advanced user pro-tip: Instead of including the signature in with the JSON object, do this: create your JSON object as usual, convert to a string, then base64-encode it. Then HMAC that. Then your token is "$base64EncodedJson:$signature". The colon won't be in either the base64 or HMAC output, so it's a good delimiter. Then the very first thing you do when validating the token is to split on the colon and make sure the HMAC of the part before the colon matches the value after the colon. If not, bail out right away. This is actually slightly less likely to be fucked up since you don't even have to decode the base64-encoded string to validate it.



  • @pjt33 said:

    If it's monumentally unlikely that an attacker will break the token encryption then there's no need to include either the password or the hash: the existence of the token communicates the correctness of the password without the need for any shared session.

    I wouldn't bother encrypting the payload, myself, just HMAC it and tack the signature on the end.



  • @morbiuswilters said:

    then you should be using a session in the first place.

    At the time I implemented my system, I was using a load balancer that didn't allow for sticky sessions. (The one in AWS. It does now, I believe, however I'm not sure if sessions remain sticky even if you bring new servers online, and servers brought offline will of course lose any sessions that were "stuck" to them... so.) Sure maybe I'm a dumbhead stupidface moronmouth but I had to work with the tools available to me.



  • @blakeyrat said:

    @morbiuswilters said:
    then you should be using a session in the first place.

    At the time I implemented my system, I was using a load balancer that didn't allow for sticky sessions. (The one in AWS. It does now, I believe, however I'm not sure if sessions remain sticky even if you bring new servers online, and servers brought offline will of course lose any sessions that were "stuck" to them... so.) Sure maybe I'm a dumbhead stupidface moronmouth but I had to work with the tools available to me.

    No, that's perfectly fine, I wasn't criticizing you. I'm just saying "Use a session if at all possible". Sometimes it's not, sometimes there's a definite need for stateless tokens, but a lot of times there isn't and a session is going to do everything you need much better--and safer--than you can do on your own.

    By the way, I've used the method I describe above to generate CSRF tokens that let me embed security info in a form field. That way I didn't have to mess around with session management or with storing multiple CSRF tokens in the same session (in case the user does something crazy, like opening more than one tab..)


  • Discourse touched me in a no-no place

    @blakeyrat said:

    @morbiuswilters said:
    then you should be using a session in the first place.

    At the time I implemented my system, I was using a load balancer that didn't allow for sticky sessions. (The one in AWS. It does now, I believe, however I'm not sure if sessions remain sticky even if you bring new servers online, and servers brought offline will of course lose any sessions that were "stuck" to them... so.) Sure maybe I'm a dumbhead stupidface moronmouth but I had to work with the tools available to me.

    If you can have a shared database that holds the session data, you don't need to worry about which web server fields the user request. It's not quite as pretty a solution as trying to keep everything totally stateless, but it is relatively easy to get right. And it doesn't involve sending the users credentials around all over the place!



  • @dkf said:

    @blakeyrat said:
    @morbiuswilters said:
    then you should be using a session in the first place.

    At the time I implemented my system, I was using a load balancer that didn't allow for sticky sessions. (The one in AWS. It does now, I believe, however I'm not sure if sessions remain sticky even if you bring new servers online, and servers brought offline will of course lose any sessions that were "stuck" to them... so.) Sure maybe I'm a dumbhead stupidface moronmouth but I had to work with the tools available to me.

    If you can have a shared database that holds the session data, you don't need to worry about which web server fields the user request. It's not quite as pretty a solution as trying to keep everything totally stateless, but it is relatively easy to get right. And it doesn't involve sending the users credentials around all over the place!

    Some platforms make it easy to put sessions into a database (or something non-relational like memcached or Redis.) Others pretty much require the session to exist on a single server (or to implement complex "clustering" to share sessions.) Most of what I do involves the former, but if you're working with the latter and you can't do sticky sessions (and Blakey pointed out he was using Elastic Load Balancer which was launched half-baked, like a lot of AWS shit, and lacked sticky sessions) then you can do worse than using a token that can be authenticated on its own.


  • Discourse touched me in a no-no place

    @morbiuswilters said:

    Some platforms make it easy to put sessions into a database (or something non-relational like memcached or Redis.) Others pretty much require the session to exist on a single server (or to implement complex "clustering" to share sessions.) Most of what I do involves the former, but if you're working with the latter and you can't do sticky sessions (and Blakey pointed out he was using Elastic Load Balancer which was launched half-baked, like a lot of AWS shit, and lacked sticky sessions) then you can do worse than using a token that can be authenticated on its own.
    I totally understand. I work with stuff that can't go in a database as it's most easily modelled as a process, not a collection of rows in a DB, so load balancing that is rather tricky (it's also session-less for various reasons). But sharing the database (if you can use one, and I don't care for the purposes of this argument about whether it's relational or not) still helps a lot. Assuming that most of what you're doing is reads; write-heavy stuff needs a different approach as the DB is likely to be the bottleneck anyway.

    ELB handles a specific sort of load pattern. Heck, the whole AWS is like that: they support certain types of things only, they tell you what those are, and if you don't like it you can always go somewhere else. (If only more of the "cheaper" alternatives weren't run by morons.)



  • @blakeyrat said:

    @pjt33 said:
    If it's monumentally unlikely that an attacker will break the token encryption then there's no need to include either the password or the hash: the existence of the token communicates the correctness of the password without the need for any shared session.

    Well fair enough, but now you're arguing for a design that would make retracting tokens impossible and mine isn't that way.

    Huh? I'm pretty confident that you weren't expiring session tokens by changing users' passwords, so I don't see how not including the password in the token affects the ability to expire it.



  • There's a difference between "weren't" and "can't".



  • @blakeyrat said:

    There's a difference between "weren't" and "can't".





    I am reading a book by Michael Gates. There is one word called antithalian. I think that is blackeyrat.



  • @Nagesh said:

    blackeyrat

    wat


  • Considered Harmful

    @Nagesh said:

    Caesar encryption

    Hahaha. AHAHAHAHAHA.

    I know you're a troll but that's a nice one. Caesar encryption is weak enough to break by a kid in grade school with a pen and paper, and I'm not exaggerating.



  • @blakeyrat said:

    Where the fuck were you people like a year and a half ago when I asked advice on this? Sheesh.

    In any case, you can still invalidate the token easily enough-- whenever any of the web servers receives the token, they still have to authenticate the user. (They don't just take it on faith.) So if the bits that come back from the database say, "oh hey this guy's no longer an admin", the page will not only treat them as non-admin but the new token it generates will also be a non-admin token. So I don't see invalidation as an issue; given my app was simple and only had like 3 levels of permissions. It does make it pointless to keep data like "isAdmin" in the token if you're going to have to look it up anyway 100% of the time, but.

    The problem I was solving is that servers don't have any way of sharing sessions among themselves, so keeping authentication in a normal session cookie wasn't an option. What they *do* all have is access to the user database.

    Then why not add a table of session IDs to the user database, instead of all this fartarsing about with stuffing possibly sensitive things into tokens? If you're going to have to hit the user DB to authenticate the username and password in the token anyway, why not just authenticate a nonce?


  • I see you have as per your convenience completely missed the rest of the question, by choosing to focus on one item in the post. Are you suffering in your life?

     

    @joe.edwards said:

    @Nagesh said:
    Caesar encryption

    Hahaha. AHAHAHAHAHA.

    I know you're a troll but that's a nice one. Caesar encryption is weak enough to break by a kid in grade school with a pen and paper, and I'm not exaggerating.

     



  • @flabdablet said:

    Then why not add a table of session IDs to the user database, instead of all this fartarsing about with stuffing possibly sensitive things into tokens? If you're going to have to hit the user DB to authenticate the username and password in the token anyway, why not just authenticate a nonce?

    Each server has access to a user database. Not the user database.

    It only connected to *the* user database if it needed to create a new user.



  • @blakeyrat said:

    @flabdablet said:
    Then why not add a table of session IDs to the user database, instead of all this fartarsing about with stuffing possibly sensitive things into tokens? If you're going to have to hit the user DB to authenticate the username and password in the token anyway, why not just authenticate a nonce?

    Each server has access to a user database. Not the user database.

    It only connected to *the* user database if it needed to create a new user.

    Fair enough.

    If those were my constraints, I would have dealt with them by including a random UUID in the user database for the specific purpose of validating session tokens, rather than re-purposing the username and password. That way, if my system does suffer from leaky tokens I'm not compromising anything that could be of use for attacking any system other than mine.


Log in to reply