"secure" HTTP data-feed



  • A while ago I was working on a project to upgrade a system responsible for updating event information for a satellite television company.  This stuff is usually referred to as SI (Service Information) and these days it can be of considerable quantity, containing all manner of data necessary for viewers to know what channels are available, what's on, event blurbs etc. you get the idea.

    If you had to send this stuff across international borders at regular intervals, how would you do it?

    We used an XML payload over a HTTP/1.1 pipe.  We specified that it be over a VPN or (at least) HTTPS.  This particular client decided that a VPN was too expensive (it boggles the mind I know) and said they would be happy to have the data being sent of public lines but the job of securing the link & the web-server would go to their IT department.  Afterwards, I decided to check up on their implementation of this ideal...

    They had simply enabled "HTTP authentication" on the server.

    At this point, I should mention that the data was going to be seen by around 40 million people - assuming it could be seen at all.  Many parts of the payload are essentially metadata instructions for the backend encoders and could potentially prevent the receiving hardware from being able to figure out what the hell it meant.  If hacked, the channels would still be there but (depending on what was changed) the hardware might not be able to find them.

    Obviously, I hoped the IT department's work was just for test purposes and asked the IT guys about it.  To my amazement they thought it was perfectly secure.  Afterall, when they typed in the public address in a web-browser the browser dutifully popped-up an authentication dialog.  I began to explain that any username/password they typed in was being sent in clear-text because there was no encryption but the conversation rapidly devolved as I realised that these guys had absolutely no idea what I was talking about.  They actually became actively hostile towards me as they slowly figured out I did not share their confidence in their solution.

    I talked to my manager and we got the story out to some degree to the managers on the other side of the fence .  As I understand it their IT department continued to assert that the link was somehow secure and to this day I believe that system is still running in clear-text.

    No, I will not divulge where or who the company is (I have changed all the minor details to cover the tracks somewhat too).

    This incident is one of many security-related WTFs I've experienced but this was the only case where ignorance won and it really highlighted to me a clear security lesson that everyone should know...

    Data security consists of 2 primary parts;
    1. Encryption
    2. Authentication
    If either one is broken or not persent; the system is compromised.



  • So it was HTTPS + HTTP authentication, or just HTTP auth. ?

    First one is not THAT bad. VPN doesn't do that much above it (except for hiding the connection protocol at all in basic case).



  • [quote user="viraptor"]So it was HTTPS + HTTP authentication, or just HTTP auth. ?[/quote]

    Basic HTTP with Auth enabled (no encryption anywhere).

    [quote user="viraptor"]First one is not THAT bad. VPN doesn't do that much above it (except for hiding the connection protocol at all in basic case).[/quote]

    Agreed, though VPN is much stronger.



  • Were they doing basic HTTP authentication, or digest HTTP authentication? Digest auth is secure over plaintext, basic isn't.

    It's amazing how many people don't get that authentication != encryption. Just because you are authenticating, it doesn't mean your data is encrypted.



  • > Digest auth is secure over plaintext, basic isn't

     

    PLEASE - digest auth is NOT secure at all over plaintext! Anybody sniffing your traffic can grab the digested password and replay it anytime, with no more effort than if you had sent the password itself.

     

    Have a look at http://httpd.apache.org/docs/1.3/howto/auth.html#digestcaveat for more info. Especially the "with regard to security considerations" part.

     

     



  • [quote user="bobday"]

    PLEASE - digest auth is NOT secure at all over plaintext! Anybody sniffing your traffic can grab the digested password and replay it anytime, with no more effort than if you had sent the password itself.

    [/quote]

    Are you saying, that digest auth is just sending the same static hash every time? I'm not using it, so I was never really interested, but I always assumed, that it's kind of challenge-response algorithm. Like hashes of pass+random text from server, or whatever.

    If it's not like this, then what's the reason for digest at all?



  • Surely the solution would be to not tell anyone the url of the http server to contact for the information?

    Ta-da! Security through obscurity! 



  • [quote user="bobday"]

    > Digest auth is secure over plaintext, basic isn't

     

    PLEASE - digest auth is NOT secure at all over plaintext! Anybody sniffing your traffic can grab the digested password and replay it anytime, with no more effort than if you had sent the password itself.

     

    Have a look at http://httpd.apache.org/docs/1.3/howto/auth.html#digestcaveat for more info. Especially the "with regard to security considerations" part.

     

     

    [/quote]

    The article is incorrect.  They state that:

    "someone very familiar
    with the workings of HTTP could use that information - just
    your digested password - and use that to gain access to the
    content, since that digested password is really all the
    information required to access the web site."

    This is not correct.  The hashed password is never sent.  The hashed password is used in a second hash and that is the hash that is sent.  So while yes, you only need the hash of the password, this is never sent.  It appears that they have misunderstood digest authentication, or they are talking about more than just the transaction itself.

    Any implementation that allows replay attacks is faulty, since the nonce count must increment if you are reusing the nonce.  This requires the digest hash to change, which requires knowledge of the password.

    Digest is a secure way of authenticating over plaintext.  It is NOT encryption, only authentication, so yes, your data is visible.  If you use qop=auth-int, your data can't be tampered with, as the whole package is essential digitally signed with your password.

    The article states:

    "The moral of this is that if you have content that really
    needs to be kept secure, use SSL"

    This is correct, but misleading.  If you want secure authentication, you can use digest authentication over plaintext.  You you want secure data (very different concept), you should use SSL.

    Authentication != Encryption. 



  • [quote user="Grimoire"]

    The hashed password is used in a second hash and that is the hash that is sent....

    [/quote]

    Thanks for correction... For a second there, I lost my faith in W3C :D



  • [quote user="viraptor"][quote user="Grimoire"]

    The hashed password is used in a second hash and that is the hash that is sent....

    [/quote]

    Thanks for correction... For a second there, I lost my faith in W3C :D

    [/quote]

    You mean the IETF?

    It's unfortunate that not all servers bother to check for replayed credentials, since every page access in a protected area is supposed to use a unique salt. 



  • [quote user="foxyshadis"]

    You mean the IETF?

    [/quote]

    I mean "Network Working Group" whoever they belong to.



  • [quote user="Grimoire"]

    Digest is a secure way of authenticating over plaintext.  It is NOT encryption, only authentication, so yes, your data is visible.  If you use qop=auth-int, your data can't be tampered with, as the whole package is essential digitally signed with your password.

    [/quote]

    What the document is trying (but failing) to say is that if you don't do this, somebody can use a trivial MITM attack by intercepting your message, duplicating the digest response, and adding their own text.

    Almost nobody implements auth-int (because SSL exists, and the minor advantage of lower processing power is not considered important), and nobody did at the time the document was written - I'm not even sure it existed back then.



  • [quote user="Xarium"]

    Data security consists of 2 primary parts;

    1. Encryption
    2. Authentication

      If either one is broken or not persent; the system is compromised.

    [/quote]

    Hrngh.

    Information ("data") security consists of three components:

    1. Confidentiality
    2. Integrity
    3. Availability

    (Standard mnemonic: CIA) 

    Encryption, authentication, and authorisation are three things which together can be used to provide confidentiality and integrity. They do nothing for availability, and there are other ways to manage integrity. Encryption and authentication are useless without proper authorisation.

    A partial coverage is at least as bad as no coverage. A specific problem may not have strong requirements in some areas (you may need only integrity but not confidentiality, so no encryption is required, or you may need availability only during office hours), but most problems will have at least integrity and availability requirements.

    People who focus on encryption and authentication tend to forget availability. Classic example: systems on the internet which lock an account after three bad password attempts, thereby allowing anybody to lock the account of any user by trying three random passwords.



  • Hmm, funny thing...  neither Bruce Schneier nor Niels Ferguson have ever mentioned your "CIA" acronym in any of their security talks, presentations or books; yet they both regularly expound the 2 aspects I mentioned.

    So pardon me but I think I'll ignore your obtuse ways of saying the same thing and stick with the cryptographic experts on this matter.



  • [quote user="Xarium"]

    Hmm, funny thing...  neither Bruce Schneier nor Niels Ferguson have ever mentioned your "CIA" acronym in any of their security talks, presentations or books; yet they both regularly expound the 2 aspects I mentioned.

    So pardon me but I think I'll ignore your obtuse ways of saying the same thing and stick with the cryptographic experts on this matter.

    [/quote]

    His concepts and your concepts are orthogonal.  Confidentiality (preventing people who shouldn't have access from gaining access), integrity (preventing people who don't have permission from modifying the data), and availability (preventing people from preventing the authorized from obtaining the data) are part of the "why" of securing things, encryption and authentication are part of the "how".



  • [quote user="Xarium"]

    Hmm, funny thing...  neither Bruce Schneier nor Niels Ferguson have ever mentioned your "CIA" acronym in any of their security talks, presentations or books; yet they both regularly expound the 2 aspects I mentioned.

    [/quote]

    Just because two experts haven't ever said a particular thing to you, you assume it's not true? Just because they regularly talk about two things, you assume there is nothing else that matters? (And Schneier at least certainly does cover integrity and availability on a regular basis, although he prefers to describe threat models using more finely graded terms)

    You can find the history of the CIA mnemonic and a discussion of information security on Wikipedia, along with all the references you're likely to need.



  • [quote user="asuffield"]Just because two experts haven't ever said a particular thing to you, you assume it's not true? Just because they regularly talk about two things, you assume there is nothing else that matters?[/quote]

    Why don't you read my whole post instead of inventing what you think I'm going to say?

    Here, I'll save you the trouble of scrolling up...

    [quote user="Xarium"]So pardon me but I think I'll ignore your obtuse ways of saying the same thing...[/quote]

    What aspect of data security is not covered by Encryption & Authentication?  I have enough trouble getting people to understand those 2; so why complicate it by using 3 abstract terms that evaluate to the same result?

     



  • [quote user="Xarium"]

    [quote user="asuffield"]Just because two experts haven't ever said a particular thing to you, you assume it's not true? Just because they regularly talk about two things, you assume there is nothing else that matters?[/quote]

    Why don't you read my whole post instead of inventing what you think I'm going to say?

    Here, I'll save you the trouble of scrolling up...

    [quote user="Xarium"]So pardon me but I think I'll ignore your obtuse ways of saying the same thing...[/quote]

    What aspect of data security is not covered by Encryption & Authentication?  I have enough trouble getting people to understand those 2; so why complicate it by using 3 abstract terms that evaluate to the same result?

     

    [/quote]

    Authorisation.

    Authentication merely identifies you. Authorisation determines what you can do.

    Authentication isn't much good if, once in the system, I can change a whole bunch of data I'm not supposed to.

     


Log in to reply