Apple Networking Technology



  • I recently learned more about Apple networking technology than I ever wanted to know... This is probably too intricate to be front-page material, but there are really big WTFs in here, so I thought I'd share it

    The environment: mixed set of Windows, Linux and Mac OS X clients; all clients authenticate against either an NT domain controller (Windows) or Kerberos KDC (Linux/Mac);users have a shared home directory for all clients off a central fileserver, served via SMB (Windows) or NFS (Linux/Mac)

    The task: "tighten up" the less-than-secure file serving: anyone able to connect to the network could hijack an SMB connection, or simply pretend to be another user to fool the NFS service. In effect a malicious attacker could read and modify arbitrary files.

    Okay, what I did... Step 1: enable SMB signing. This thwarts connection hijacking attacks and makes Windows file serving secure. An eavesdropper can still read what is written to the fileserver, but he cannot interfere. NT authentication is already secure, so I am done with that.

    Step 2: Enable Kerberos NFS mounts on the Linux clients; since everything else is already in place this is as easy as adding the mount option "sec=krb5i" which provides the same security as SMB signing; I *could* use krb5p to encrypt the data over the network, but since the Windows clients are already communicating unencrypted it buys nothing (except slowing things down a lot).

    Step 3: Now we take care of the Macintoshes.

    Attempt 1: Use Kerberized NFS; the Apple man-page states that the switch "-k" activates Kerberos security for NFS mounts. However it does not work. A little web search later it turns out that it is *konwn* not to work. Actually, the feature is intentionally deactivated because it does not work, despite being prominently documented. (WTF #1)

    Attempt 2: Well, Macintoshes are great and can communicate using any existing protocol on the planet (or so Apple tells me), consequently the next logical thing to try is SMB, as the Windows clients already use it. At first it looks promising, since I am able to access a test SMB share from the Macintoshes without hassle if I access it manually *after* I have logged in and have my home directory mounted already. However using SMB to access the home directories fails spectacularly - for starters, it does not work if SMB signing is activated (WTF #2), and even if deactivated, some apps cease to work at all (sorry, don't remember which, but nevertheless WTF #3). Reading the Apple documentation tells me that using SMB for the home directories is discouraged, and I begin to understand why. So this is a complete no-go.

    Attempt 3: The protocol recommended by Apple is AFP. At first I am a little bit concerned that performance may suck (because AFP is a little bit aged), but after setting up netatalk and successfully connecting to a test share, I am pleasantly surprised that it is on par with NFS and SMB. Now I go setting it up for real; netatalk has a Kerberos module, which is neat because we are using Kerberos already, so I activate it. Configuring the Macintoshes to automount the AFP share at user login requires a bit documentation browsing but is straightforward otherwise.

    Logging in with my password works and yields my home directory. Yeah, it appears I am done!

    Just to finish up everything I start poking around in the log files to verify that everything is working. Hm, very odd: netatalk tells me that the Macintosh clients are using clear-text passwords to connect to the AFP share (send the clear-text password over to the AFP server, which in turn verifies it with the Kerberos server), even though Kerberos is configured and available. Confused, I (manually) connect to an AFP share, and sure enough the Mac uses secure Kerberos authentication against the AFP server, instead of insecure plaintext.

    After a little bit of further experimentation, I am convinced that this is the rule: Manually connect to an AFP-share: can use secure Kerberos authentication. Auto-connect to an AFP-share (which *has* to be done for the home directory): can only use insecure plain-text authentication. So what happens when I log in to one of the Macintoshes: The Computer verifies my password using the cryptographically secured Kerberos mechanism; it obtains tickets for me to access further services; it then sends my password in plain-text over the wire to the file server to access my home directory (OMGWTF #4).

    At this point I am already at a loss of words, but the security of this setup is as good as it gets given the constraints I had, so it was deemed acceptable. However the story is not finished yet... After the setup is deployed, a small number of users complain that they cannot login to the Macintoshes anymore (Windows and Linux clients work just fine). The error message they got when trying to login was that the AFP server could not verify the password (despite the password being evidently correct), and in fact the log files of the AFP server clearly said "wrong password".

    As it turns out, the AFP protocol does not allow more than 8 characters per password... So the login procedure goes as follows: verify the (full-length) password against the Kerberos server, truncate it to 8 characters, send it to the AFP server, which in turn notices that the truncated password does not match the full-length password, d'oh. I think there are two distinct WTFs here: Allowing only 8 character passwords in a protocol in the year 2006 (WTF #5), and then giving a misleading error message instead of warning about the likely problem (WTF #6).

    The story ends here, with everyone busily changing their passwords to a shorter one. 



  • I don't have anything constructive to say, but I enjoyed this story.  Especially the plaintext WTF sitting there in the middle of all your security.



  • <nice> my password is easy to hack, read and maliciously available to "An eavesdropper can still read what is written to the fileserver, but he cannot interfere" <nice/>



  • I'm guessing that you're using Windows as the AFP server, since there's a section in the netatalk documentation about this problem, right? AFP does indeed allow encrypted passwords and passwords longer than 8 characters, but the plaintext passwords problem (which also limits things to 8 characters) is a known issue with Services For Macintosh -- in short: in order to use the default encrypted password scheme, the server has to know the cleartext password; Windows uses one-way encryption by default, so this is not allowed; the fallback cleartext scheme (which is primarily meant for backwords compatability with old systems) only allows 8 characters. Voila!


    Anyway, it appears there is a solution, for once: apparently, you MUST install the Microsoft UAM on the Mac clients, and use it as the authentication method.



  • Consider yourself lucky: you appear to be working on a recent version of MacOS.  The older versions have even more WTFs:  For example, how would you change the IP address of a computer running MacOS X Server 10.2?







    The correct answer: Format and reinstall.



  • [quote user="Carnildo"]Consider yourself lucky: you appear to be working on a recent version of MacOS.  The older versions have even more WTFs:  For example, how would you change the IP address of a computer running MacOS X Server 10.2?





    The correct answer: Format and reinstall.

    [/quote]

    Holy Crapoli. OI true, that is without a doubt the lamest thing I've ever heard of from a server. Even an "FTP all user passwords used to ftp.evilhaxors.com" option would make more sense.



  • I never worked on macs, but I remember this guy I worked with who had a G3 with os 9 (this was quite some time ago) and when he wanted to hook up the modem he had to disable the network as the two were mutually exclusive - or better said, you could just run one network interface at a time. And he would insist macs were so good - they're full of WTF's like everything else.

     



  • [quote user="RayS"][quote user="Carnildo"]Consider yourself lucky: you appear to be working on a recent version of MacOS.  The older versions have even more WTFs:  For example, how would you change the IP address of a computer running MacOS X Server 10.2?





    The correct answer: Format and reinstall.

    [/quote]

    Holy Crapoli. OI true, that is without a doubt the lamest thing I've ever heard of from a server. Even an "FTP all user passwords used to ftp.evilhaxors.com" option would make more sense.

    [/quote]

    It's a fascinating idea, but not true. See

    As for the Mac OS 9 thing: true, for a given value of true. The networking code in Mac OS 9 (which was originally called Open Transport before it became part of the regular installation) was capable of handling multiple network connections simultaneously. But for some reason (probably a "might confuse the users" issue although of course I don't know) they never built an interface for it. Instead, they built an interface where you had multiple possible configurations, and could swap between them, each with one connection. There was some third party program -- I think it was IPNetRouter, although it's been a while -- you could get which allowed you to connect to multiple networks, but it wasn't free (in part because it did a lot more than just this) so most people just kept swapping back and forth, or else bought an external device to handle the networking.

    And why do I know all this? Because I held a job as IT person in a mostly-Mac shop around the time Mac OS X (client) was in its final development versions, so I have a lot of Mac OS 9 trivia rattling around.



  • [quote user="The Vicar"]I'm guessing that you're using Windows as the AFP server, since there's a section in the netatalk documentation about this problem, right? AFP does indeed allow encrypted passwords and passwords longer than 8 characters, but the plaintext passwords problem (which also limits things to 8 characters) is a known issue with Services For Macintosh -- in short: in order to use the default encrypted password scheme, the server has to know the cleartext password; Windows uses one-way encryption by default, so this is not allowed; the fallback cleartext scheme (which is primarily meant for backwords compatability with old systems) only allows 8 characters. Voila![/quote]

    No, the netatalk server(s) are running on Linux, though running them on Windows would not make a difference in this case. To use the hashed (or "encrypted" though this is a misnomer) authentication schemes in AFP, I would have to expose a complete list of plaintext passwords to the file server... errr, to *every* file server to be precise... Thinking for a moment about this I cannot seriously consider this as a good idea.

    Not to mention that up until now the clear text password is not available *anywhere*, not even on the authentication servers, neither the Windows DC nor the Kerberos server store plaintext passwords; at the very least everyone would have to reenter their password. However since the ultimate goal is to lessen the burden on the network administratos, the idea is to unify the authentication into a single AD or Samba4+Kerberos database at one point in time; adding a third (and plain-text) password database is contra-productive.

    Also I am not sure that installing additional UAMs would help at all -- mind you there is already a Keberos UAM *installed* that has also been verified working, but it is simply not *used* for the home directories (heck, I think that even uams_dhx is available...). The sheer idiocy of this still drives me mad.

    There are other annoying minor WTFs to the story I didn't mention -- for example ssh login to the Macintoshes is unuseable -- the (sort of) authenticated AFP automount mechanism for the home directories only works for local graphical login. So you can have only one of the following: remote access ability via ssh; or a file server with (sort of) authentication. But never both.



  • [quote user="caleridas"]

    [quote user="The Vicar"]I'm guessing that you're using Windows as
    the AFP server, since there's a section in the netatalk documentation
    about this problem, right? AFP does indeed allow encrypted passwords
    and passwords longer than 8 characters, but the plaintext passwords
    problem (which also limits things to 8 characters) is a known issue
    with Services For Macintosh -- in short: in order to use the default
    encrypted password scheme, the server has to know the cleartext
    password; Windows uses one-way encryption by default, so this is not
    allowed; the fallback cleartext scheme (which is primarily meant for
    backwords compatability with old systems) only allows 8 characters.
    Voila![/quote]

    No, the netatalk server(s) are running on Linux,
    though running them on Windows would not make a difference in this
    case.

    [/quote]

    Whoops! Sorry, I misread your initial post. Thought you were hosting on Windows boxes, and using Microsoft's notorious (and soon-to-be-discontinued, I'm told) Services For Macintosh. So you're right, an additional UAM won't help. (And here I thought I was helping you. Sorry!)

    After doing some more looking around with the aid of Google, I can see a few options which may or may not be work for you. On the off chance that something might help, and that you haven't already tried these ideas:

    First option: try AFS instead of AFP -- OpenAFS is free, available for both the server (Linux) and the clients (Mac OS X), and supports Kerberos. (Of course, netatalk also supports Kerberos, and still ended up with cleartext passwords, so that may not help. Still, it can't hurt to at least try, right? It looks like AFS, unlike AFP, has no fallback to cleartext passwords, so maybe it is a bit safer... at least it would just plain fail instead of creating a security hole.) In a sense, doing this would be overkill, since you wouldn't be using the features that AFS was designed for, but if it works, then who cares?

    Second option: if you know a low-level Mac geek who is willing to help, or are willing to become one yourself, compile the Darwin NFS code (which apparently has the -K option working), make an installer package, and have your users install it, then connect with Kerberized NFS. Although there may be a reason why Kerberos support in NFS didn't make it into the commercial distribution. (It's in ftp and AFP, so maybe the NFS code is shaky?)

    Third option: find out if there's a way to pass authentication for the Macs on to a copy of Mac OS X Server in a way that won't suck (and won't involve using Mac OS X Server to serve the home directories -- the mach kernel doesn't currently make a good file server for large numbers of users; it hits a performance wall because the kernel only has one IO thread; I'm told they're working on it, but right now it isn't a good idea) and if so, buy a copy and use it. (Or buy an XServe, which comes with a copy.)

    Fourth option: go with SMB and buy copies of ADmitMac for everyone. (This would cost more than getting a copy of Mac OS X Server, unless you have no more than 4 Mac users, but it would let you use signed SMB.)

    Fifth option: go to the Apple discussion boards (discussions.apple.com) and post the problem, and see if anyone else already solved it. (Presumably, there's something different in the way that networked home directories are mounted as opposed to AFP mounts after login, and there may be a workaround/fix.)

    Tied for last place: wait and see if Leopard (10.5) fixes SMB or Kerberized NFS support; use (ugh) WebDAV instead of AFP (note: WebDAV on Mac OS X apparently has some serious issues when the number of files gets large AND I'm not sure it works for home directories, but if nothing else works and you're willing to look into options...).

    As for ssh login to the Macs not being useable: what, exactly, do you mean? Are you saying "I can't log in with ssh from another machine TO the Mac"? I've never had that problem.



  • [quote user="The Vicar"]

    First option: try AFS instead of AFP --
    OpenAFS is free, available for both the server (Linux) and the clients
    (Mac OS X), and supports Kerberos. (Of course, netatalk also supports
    Kerberos, and still ended up with cleartext passwords, so that may not
    help. Still, it can't hurt to at least try, right? It looks like AFS,
    unlike AFP, has no fallback to cleartext passwords, so maybe it is a
    bit safer... at least it would just plain fail instead of creating a
    security hole.) In a sense, doing this would be overkill, since you
    wouldn't be using the features that AFS was designed for, but if it
    works, then who cares?
    [/quote] 
    Yes I did consider AFS (OpenAFS) a few years back; maybe things have improved but what I learned back then:

    - the OpenAFS server stores all files in their own container, not a regular filesystem; this means that you cannot get at the files via any means other than AFS, so *everyone* would have to use AFS
    - this means you need an AFS client on the AFS server to access the files it is serving 
    - OpenAFS was dog slow; something like 5-6MByte/s read/write speed; NFS,SMB,AFP easily saturate Fast Ethernet at ~11MB per second, and I can get ~50MB/s on GigE
    - the OpenAFS client was buggy and crash-prone; especially on mulit-CPU systems the client would lock-up within hours; did I mention that you need an AFS client on the AFS server?
    - the AFS permission model sucks 
    - specifically using AFS for the Mac home directories was *not* possible -- every solution that I had found for Macs accessing AFS involved running some custom login scripts programs from their home directory, catch-22
    So based on past experience I did not even consider AFS.
    [quote user="The Vicar"]
    Second option: if you know a low-level Mac geek who is willing to help, or are willing to become one yourself, compile the Darwin NFS code (which apparently has the -K option working), make an installer package, and have your users install it, then connect with Kerberized NFS. Although there may be a reason why Kerberos support in NFS didn't make it into the commercial distribution. (It's in ftp and AFP, so maybe the NFS code is shaky?)
    [/quote]
    It's in FTP and in AFP is not a good reason -- both FTP and AFP authenticate only "connecting to a share"; this means that after authentication is done, you can forget about it. NFS is very very very different in that the "connection to the share" is not authenticated, but every individual file access.
    I hope that Apple understandes their own NFS implementation, if it is deactivated I trust it is for a reason. As much as I want security I cannot treat the users as guinea pigs and have to provide a solution that does not leave their workstations crashing all the time.
    [quote user="The Vicar"]
    Fourth option: go with SMB and buy copies of ADmitMac for everyone. (This would cost more than getting a copy of Mac OS X Server, unless you have no more than 4 Mac users, but it would let you use signed SMB.)
    [/quote]
    Maybe a realistic option, but for this year there is no budget anymore. Maybe next year. Actually I hope that Apple simply fixes the Kerberos AFP authentication (I am writing a problem report).
    [quote user="The Vicar"]
    As for ssh login to the Macs not being useable: what, exactly, do you mean? Are you saying "I can't log in with ssh from another machine TO the Mac"? I've never had that problem.
    [/quote]
    No. I could log in to the machine using ssh. But I did not get my network home directory. Okay, I can the manually mount all shares that I want, including the share containing my home-directory, but it is kind of inconvenient -- it is not recognized as my home directory because it was not present at login time. This means that applications do not find their preferences etc. which makes working remotely quite awkward. Of course there is also the matter of what happens if two users want the same share at the same filesystem location, but with different permissions -- in short, the AFP model does not support that (NFS could, though).


  • [quote user="The Vicar"][quote user="RayS"][quote user="Carnildo"]Consider yourself lucky: you appear to be working on a recent version of MacOS.  The older versions have even more WTFs:  For example, how would you change the IP address of a computer running MacOS X Server 10.2?





    The correct answer: Format and reinstall.

    [/quote]

    Holy Crapoli. OI true, that is without a doubt the lamest thing I've ever heard of from a server. Even an "FTP all user passwords used to ftp.evilhaxors.com" option would make more sense.

    [/quote]

    It's a fascinating idea, but not true. See
    [/quote]

    That script was created shortly before 10.2 was replaced by 10.3, it's unsupported by Apple, and it's unreliable.  The only method that works 100% of the time is to re-install the OS.



  • [quote user="caleridas"]

    - the OpenAFS client was buggy and
    crash-prone; especially on mulit-CPU systems the client would lock-up
    within hours; did I mention that you need an AFS client on the AFS
    server?
    [/quote]

    Yuck. That's enough of an answer, right there.

    [quote user="caleridas"]
    I hope that Apple understandes their own NFS implementation, if it is deactivated I trust it is for a reason. As much as I want security I cannot treat the users as guinea pigs and have to provide a solution that does not leave their workstations crashing all the time.
    [/quote]

    It may be deactivated for some inscrutable reason that has nothing to do with stability. Apple has, in the past, allowed unstable features to make it into the Mac OS X commercial release, as long as they aren't part of the ordinary GUI operation. Case in point: support for sparse disk images (which, combined with encryption, are the foundation of FileVault) was added to the OS a major revision before Apple made it possible to use them via the GUI; you had to read the hdiutil man page to figure out how to make one. These early sparse images were terribly prone to corruption (the problems weren't fixed until one of the last revisions before the GUI added support), but you could still make them from the command line with a standard install of Mac OS X, and once you made them they acted like any other disk image. So when it comes to a feature like Kerberos support in NFS, which is really not in the GUI and wouldn't be used by 99.9% of users, there's no real relation between what's available and what isn't.

    [quote user="caleridas"]
    [quote user="The Vicar"]
    Fourth option: go with SMB and buy copies of ADmitMac for everyone. (This would cost more than getting a copy of Mac OS X Server, unless you have no more than 4 Mac users, but it would let you use signed SMB.)
    [/quote]
    Maybe a realistic option, but for this year there is no budget anymore. Maybe next year. Actually I hope that Apple simply fixes the Kerberos AFP authentication (I am writing a problem report).
    [/quote]

    Don't just submit a problem report, go to the discussion board I mentioned and post something there. If it's an issue that Apple will fix in 10.5, then they may never issue a fix for 10.4 and 10.3. Whereas if someone can even tell you what the difference is between AFP-mounted home directories and ordinary AFP mounts, you may be able to figure out a fix.

    [quote user="caleridas"]
    [quote user="The Vicar"]
    As for ssh login to the Macs not being useable: what, exactly, do you mean? Are you saying "I can't log in with ssh from another machine TO the Mac"? I've never had that problem.
    [/quote]
    No. I could log in to the machine using ssh. But I did not get my network home directory. Okay, I can the manually mount all shares that I want, including the share containing my home-directory, but it is kind of inconvenient -- it is not recognized as my home directory because it was not present at login time. This means that applications do not find their preferences etc. which makes working remotely quite awkward. Of course there is also the matter of what happens if two users want the same share at the same filesystem location, but with different permissions -- in short, the AFP model does not support that (NFS could, though).
    [/quote]

    Yep, that's right. (At least, when the users are both working through the same machine.) The mount is associated with a particular user/password, which serves as the authentication on the server, so two users both connected through the same machine will see the same thing.

    You could possibly figure out some way to change the default mount point to somewhere within the home directory, which would allow different users to mount to what is nominally the same position, something like ~/Volumes/, and then have people mount manually (or set up a shell script and have them add it to their login).


  • [quote user="Carnildo"][quote user="The Vicar"]


    It's a fascinating idea, but not true. See
    [/quote]

    That script was created shortly before 10.2 was replaced by 10.3, it's unsupported by Apple, and it's unreliable.  The only method that works 100% of the time is to re-install the OS.
    [/quote]

    Dunno. Is two and a half months "shortly before"? The script definitely was hosted by Apple at least that long before 10.3 came out. And if it's as unreliable and unsupported as you claim, why is it hosted on Apple's support site?

    After reading a bit more about the problem, what I'm seeing is this: if you were dumb enough to tell the setup assistant that the IP address was never going to change, and then changed the IP address, stuff breaks. (Note that if you told it the IP address would change, this problem doesn't occur.) Wow, what a surprise! Hey, you know, I cemented my wall-mount phone into the wall, and when I ripped it out of the wall to move it to another room it stopped working! That just goes to show, wall-mount phones don't work!



  • [quote user="The Vicar"][quote user="Carnildo"][quote user="The Vicar"]


    It's a fascinating idea, but not true. See
    [/quote]

    That script was created shortly before 10.2 was replaced by 10.3, it's unsupported by Apple, and it's unreliable.  The only method that works 100% of the time is to re-install the OS.
    [/quote]

    Dunno. Is two and a half months "shortly before"? The script definitely was hosted by Apple at least that long before 10.3 came out. And if it's as unreliable and unsupported as you claim, why is it hosted on Apple's support site?
    [/quote]
    I suspect it's hosted there because users were screaming for some way to change the IP address for their servers.  It certainly wasn't part of the original OS install.  If you read the page, you'll see that Apple won't provide unpaid support for the script, and the unreliability comes from personal experience.

    After reading a bit more about the problem, what I'm seeing is this: if you were dumb enough to tell the setup assistant that the IP address was never going to change, and then changed the IP address, stuff breaks. (Note that if you told it the IP address would change, this problem doesn't occur.) Wow, what a surprise! Hey, you know, I cemented my wall-mount phone into the wall, and when I ripped it out of the wall to move it to another room it stopped working! That just goes to show, wall-mount phones don't work!

    It does make for quite an unwelcome surprise when you try to fix the WTF-worthy network setup your predecessor had created, and upon rebooting the company's fileserver after changing the IP address (once you leave that setup wizard, there's no indication that the IP is permanent), you find that it's completely unresponsive: attempts to log in from the keyboard give you an "invalid password" error, and network logins hang forever.  Diagnosing and fixing the problem made for some very long nights, since the only way to recover from a failed change was to restore the OS drive from the previous day's backup tapes.



  • [quote user="caleridas"]
    Maybe a realistic option, but for this year there is no budget anymore. Maybe next year. Actually I hope that Apple simply fixes the Kerberos AFP authentication (I am writing a problem report).
    [/quote]

    Sorry, I still have technical support reflexes, and since I'm a Mac user your problem triggers them. So I'm back with another crack at the problem. You may already have investigated this, but what the heck:

    It looks like, when the share point is remounted, it uses a URL which it pulls from the home_loc attribute passed back from the directory server. (Or otherwise built... your system doesn't use Mac OS X Server, so I have no idea where, exactly, things are coming from. But you're dealing with the keys "home_loc" and "home".)

    My latest guess is that your users' "home_loc" values look something like:
    afp://server_address/more_stuff
    or possibly:
    afp://user@server_address/more_stuff

    If this guess is right, then either your server isn't properly reporting its supported UAMs (it's either leaving Kerberos out altogether or giving it an incorrect value in the reply to FPGetServerInfo), or else the AFP mounter doesn't try Kerberos unless you tell it do so. (In which case, it's another WTF, because the documentation says that AFP will try to use the most secure UAM it can, and Kerberos certainly beats cleartext.)

    The solution, once again conditional on this guess being right, would be to alter the "home_loc" url to
    afp://;AUTH=Client%20Krb%20v2@server_address/more_stuff
    or
    afp://user;AUTH=Client%20Krb%20v2@server_address/more_stuff
    depending on which one you were using before.

    Another possibility to try would be to disable all the UAMs in netatalk except Kerberos. (That may, however, not be possible in your setup. It's definitely something netatalk can do, though, so keep it in mind.)


  • Oh, did I mention that (as you are probably aware, but it never hurts to check) that all AFP share points which contain network home directories now have to be automountable? (That appears to be a synonym for "must support guest connections".)


    Which is stupid, and appears to be new in 10.4. But I suspect you already have a setup which does this, so I forgot to mention it last night.




  • [quote user="The Vicar"]

    My latest guess is that your users' "home_loc" values look something like:
    afp://server_address/more_stuff
    [/quote]
    correct 
    [quote user="The Vicar"]
    afp://user@server_address/more_stuff

    If
    this guess is right, then either your server isn't properly reporting
    its supported UAMs (it's either leaving Kerberos out altogether or
    giving it an incorrect value in the reply to FPGetServerInfo), or else
    the AFP mounter doesn't try Kerberos unless you tell it do so. (In
    which case, it's another WTF, because the documentation says that AFP
    will try to use the most secure UAM it can, and Kerberos certainly
    beats cleartext.)

    [/quote]
    The server *does* report Kerberos as an option. In fact if I connect to the share manually (and enter the exact same afp URL) I am explicitly prompted for my "Kerberos password", there is no warning about using plaintext (which is shown otherwise), and I can see in the server log that uams_gss is used
    [quote user="The Vicar"]
    The solution, once again conditional on this guess being right, would be to alter the "home_loc" url to
    afp://;AUTH=Client%20Krb%20v2@server_address/more_stuff
    or
    afp://user;AUTH=Client%20Krb%20v2@server_address/more_stuff
    depending on which one you were using before.
    [/quote]
    this does *not* make any difference; plaintext authentication is used nevertheless 
    If i remove uams_clrtxt from the list of authenticators on the server, then it uses anonymous access instead (and hence I cannot write anything and not read any of my private files)
    If i remove uams_guest, then login is refused completely
    I guess I could list this as WTF#8 (not honouring the authentication policy set by the administrator, and silently falling back to insecure plaintext without warning whatsoever)
    [quote user="The Vicar"]
    Another possibility to try would be to disable all the UAMs in netatalk except Kerberos. (That may, however, not be possible in your setup. It's definitely something netatalk can do, though, so keep it in mind.)

    [/quote]

    Does not work, see above.

    Sorry, but the handling of network home directories on Apple systems is simply FUBAR.



  • [quote user="caleridas"]

    Sorry, but the handling of network home directories on Apple systems is simply FUBAR.

    [/quote]

    Not really, they just expect you to be running Mac OS X Server as the home directory server to do this particular trick. Similar situations exist (or at least did last time I had to do anything along those lines) with Windows and SAMBA, but you aren't running into them. There's a lot of documentation on Apple's site about how to set up Kerberized access to AFP home directories, and all of it involves using their configuration tools. You're using an someone else's server and, in effect, trying to mimic Apple's product without having a copy around to compare. There's nothing wrong with that, it just means you need to understand Apple's software a lot more thoroughly than it sounds like you do. (Which, incidentally, means that if you ever get it working, you could do a huge favor to netatalk by publicly describing what has to be done.)

    From what you've said, it sounds like Kerberos isn't being used because it can't be used at that point, which suggests that the afp mounting software is somehow unaware that it has a Kerberos key. That suggests, in turn, that there is an LDAP (or whatever) key missing, or possibly a missing/incorrect configuration file on the clients. (I found a PDF via Google of a presentation from Apple about authentication that mentions something about all the clients having "the right plist for Kerberos" at login time, but unfortunately that's all it says.) Have you tried finding an institution that actually uses Mac OS X Server and Kerberos and asking them what keys are returned by the authentication server?

    I did a little searching, and it appears that there's a guy at Indiana University who probably can help -- he figured out how to get Mac OS X Server to substitute a third-party KDC for the built-in one, which suggests that he understands all the pieces of the problem well enough to tell you what needs to be present at login to make things work. I bet he can help you:



  • [quote user="The Vicar"]

    [quote user="caleridas"]

    Sorry, but the handling of network home directories on Apple systems is simply FUBAR.

    [/quote]

    Not really, they just expect you to be running Mac OS X Server as the home directory server to do this particular trick. Similar situations exist (or at least did last time I had to do anything along those lines) with Windows and SAMBA, but you aren't running into them. There's a lot of documentation on Apple's site about how to set up Kerberized access to AFP home directories, and all of it involves using their configuration tools. You're using an someone else's server and, in effect, trying to mimic Apple's product without having a copy around to compare. There's nothing wrong with that, it just means you need to understand Apple's software a lot more thoroughly than it sounds like you do. (Which, incidentally, means that if you ever get it working, you could do a huge favor to netatalk by publicly describing what has to be done.)
    [/quote]
    Well since this is all about client-side configuration and not server-side configuration I fail to see the point. The "use Mac OS X server" argument does not cut it because it does not change client-side behaviour in any way. As for the myth that OS X server could be doing something different magically -- no, not the case; after all OS X server comprises of MIT Kerberos (guess what we used) and OpenLDAP (guess what we used); the only difference would be netatalk, but a simple network protocol dump shows that it is not behaving any differently (and why do you assume I did not have a copy of OS X server to toy around with?) 
    Well, I received a private response that what I am trying to do is simply not (yet) possible; the "mnthome" command internally used to mount the home directory can accept a password, but not a kerberos ticket; at the moment I cannot verify the veracity of this statement.
    However even assuming Kerberos authentication worked as it should, AFP is a joke compared to NFS and SMB, security-wise. It is trivial to "hijack" an AFP connection from another computer on the net after authentication is completed, which is *not* possible with NFS or SMB (with Kerberos security and signing activated, respectively). The handling of the Kerberos service principal name in AFP is an invitation for a man-in-the-middle attack (the client tells the server "Who are you?"; and after that, if the server responds "I am host/evilmachine@REALM" the client obediently authenticates to this principal; if you understand how Kerberized services "normally" work you can appreciate the full WTFness). The impression I get is that all of this was written to satisfy a "bullet-point list" of features without anyone understanding the real issues.
    Just in case it is not obvious... I have strong security background (which is why I am doing this after all) and thus a very different perspective from most other people: I care only very little about anything else than sound security policies and their correct implementation. And if a given piece of software prevents implementation of a sound policy, it has to be scrapped :)
    [quote user="The Vicar"]
    From what you've said, it sounds like Kerberos isn't being used because it can't be used at that point, which suggests that the afp mounting software is somehow unaware that it has a Kerberos key. That suggests, in turn, that there is an LDAP (or whatever) key missing, or possibly a missing/incorrect configuration file on the clients. (I found a PDF via Google of a presentation from Apple about authentication that mentions something about all the clients having "the right plist for Kerberos" at login time, but unfortunately that's all it says.) Have you tried finding an institution that actually uses Mac OS X Server and Kerberos and asking them what keys are returned by the authentication server?
    [/quote]
    No in fact not; the vast majority is using unauthenticated NFS (because it is so easy to setup, I presume); all others use local home directories with authenticated network shares mounted after login (which works without trouble); one site is using AFS, but not as primary home directory (they say it cannot be made working); one site is using AFS home directories and a Mac OS X server, but with plaintext authentication (they didn't know it was plaintext until I showed them network dumps).
    [quote user="The Vicar"]
    I did a little searching, and it appears that there's a guy at Indiana University who probably can help -- he figured out how to get Mac OS X Server to substitute a third-party KDC for the built-in one, which suggests that he understands all the pieces of the problem well enough to tell you what needs to be present at login to make things work. I bet he can help you:
    http://www.afp548.com/filemgmt_data/files/Customizing%20Open%20Directory.pdf

    [/quote]

    Well I know most of what is written there, though it is not relevant. My problem is about client configuration, and if you read the PDF, it deals with using the Apple management tools with a preexisting LDAP and Kerberos server. It has nothing to do with authenticated home directories. Actually, if you read carefully: "... so each user's homeDirectory attribute is set to /Users/username. This forces the creation of home folders on the local hard drive." :)



  • You're missing my point.


    You know that the clients are capable of connecting via Kerberized AFP, so the problem is not support for Kerberized AFP. There is also nothing preventing the clients from connecting to the server for home directories in a generalized way (as, for example, there is with NFS: nfs_mount has the -K flag disabled). Apple's documentation actually says you can use Kerberized AFP for network home directories, so what you are trying to do should be possible.

    The solution must be some difference between what your systems are doing and what systems using Apple's built-in system do. So, what could that be?

    There is nothing special about a "home directory" connection in AFP (as opposed to other connections). So if your clients can get a Kerberized AFP connection at all from netatalk, the problem does not lie with netatalk.

    There are only three real possibilities:

    1) In order to use Kerberized AFP for network home directories, the clients require some special non-obvious static configuration by the administrator, which either comes with Mac OS X Server or is documented in the instructions there

    2) In order to connect AFP network home directories, the clients require some special key from the LDAP (or other authentication) server; Mac OS X Server probably automatically sets this special key up for you when you use the integrated services

    3) Both

    In all three cases, whatever is necessary would be something that someone with Mac OS X Server would be able to detect for you if you asked them. That's why I'm suggesting you contact this guy at Indiana University, since he obviously has studied Mac OS X Server with respect to Kerberos; he probably already knows what you need to do.



  • [quote user="The Vicar"]You're missing my point.


    You
    know that the clients are capable of connecting via Kerberized AFP, so
    the problem is not support for Kerberized AFP. There is also nothing
    preventing the clients from connecting to the server for home
    directories in a generalized way (as, for example, there is with NFS:
    nfs_mount has the -K flag disabled).
    [/quote]
    I
    think you don't understand the difference between AFP and NFS: With NFS
    you setup a static mount (or, at least a mount independent from any
    user login) to the server and that's it. So nothing has to be done at
    login time., just point to the right directory, and the user has all
    his data ready waiting for him. For this reason NFS also works well
    with ssh logins, because multiple users can have directories
    accessible to them mounted simultaneously and in the same place. With
    AFP you have to perform the mount operation at login time. So there is
    a very very big difference in using the two.

    [quote user="The Vicar"] Apple's documentation actually says you can use Kerberized AFP for network home directories, so what you are trying to do should be possible.[/quote]
    Actually I fail to find a website that explicitly states that. The documentation states that you can use AFP with Kerberos; and it states that you can use AFP for home directories. The combination of both seems implied, but maybe it is wrong.
    [quote user="The Vicar"]The solution must be some difference between what your systems are doing and what systems using Apple's built-in system do. So, what could that be?[/quote]
    Errm... maybe they just silently use plaintext authentication with no one noticing? You see, one OS X administrator I know of was quite surprised when I showed him the network protocols dumps...
    [quote user="The Vicar"]There is nothing special about a "home directory" connection in AFP (as opposed to other connections). So if your clients can get a Kerberized AFP connection at all from netatalk, the problem does not lie with netatalk.[/quote]
    There are several things special about home directory mounts (not from the server or protocol point of view, but by how it is facilitated on the client). Since the home directory can be mounted anywhere, and users are normally not permitted to mount arbirtrary shares anywhere beside the Network folder, the operation is performed with a different privilege set. I *think* mnthome reroutes the mount request through the automounter; this implies that mnthome has to pass any authentication tokens through to the automounter which then performs the operation "on behalf of" the user (but not in the user's privilege context).
    Things are completely different if you "manually" create a mount to a share; in that case mount_afp is *not* invoked by the automounter, but by the user instead. Since it is executed immediately by the user, it trivially has all of the user's authentication tokens available.
    I hope you understand that some extra work has to be performed for home directories and that there *is* in fact a difference. Maybe the automounter cannot deal with a Kerberos credentials cache, maybe the passing of the user credentials cache to the automounter through mnthome has not been implemented. Maybe both would be supported, but mnthome does not pass the credentials for some other reason, but at the moment I cannot tell. Of course no error message is logged...
    [quote user="The Vicar"]
    There are only three real possibilities:

    1) In order to use Kerberized AFP for network home directories, the clients require some special non-obvious static configuration by the administrator, which either comes with Mac OS X Server or is documented in the instructions there
    [/quote]
    But how is the mere presence of an OS X server on the network going to change the client configuration?
    [quote user="The Vicar"]
    2) In order to connect AFP network home directories, the clients require some special key from the LDAP (or other authentication) server; Mac OS X Server probably automatically sets this special key up for you when you use the integrated services
    [/quote]
    Ouch... LDAP is *not* an authentication service (though it can be (ab)used as one, it is a terribly bad idea). It is a directory service. As for the fields in the LDAP directory... they are documented reasonably well on the Apple website. Besides the fact that I *know* what OS X server puts in there, and it is not much different. Im am pretty certain that there is nothing to configure on the server side.
    [quote user="The Vicar"]
    3) Both

    In all three cases, whatever is necessary would be something that someone with Mac OS X Server would be able to detect for you if you asked them. That's why I'm suggesting you contact this guy at Indiana University, since he obviously has studied Mac OS X Server with respect to Kerberos; he probably already knows what you need to do.

    [/quote]

    Maybe he knows something about the client configuration that I don't, I will ask him before I revisit the setup in January. But just believe me two things: There *are* no knobs about the server-side Kerberos configuration to turn. There *are* no knobs about the server-side LDAP configuration to turn. I know these things damn well since I partially implemented the protocols.

    I admit that there may be something missing. But what really annoyed me was the sorry state of error reporting and logging. Basically when anything goes wrong you never get to see a message *that* something went wrong, *where* it went wrong (not even to mention *why* it went wrong); getting the whole system into at least the state it is currently in required a tremendous amount of server-side diagnosing of client-side problems. I am not used to this hackish working-style.

    And I am also not used to the kind of documentation I could find... lots of extremely detailed step-by-step instruction for simple things (which I could have figured out myself, thank you), but no concise explanation of concepts that would let me logically deduce more complicated things.



  • Just to probably embarass myself, on a fresh test setup using 10.4.8
    clients it finally works, though I cannot check the difference in
    configuration at the moment.



  • [quote user="caleridas"][quote user="The Vicar"]You're missing my point.


    You
    know that the clients are capable of connecting via Kerberized AFP, so
    the problem is not support for Kerberized AFP. There is also nothing
    preventing the clients from connecting to the server for home
    directories in a generalized way (as, for example, there is with NFS:
    nfs_mount has the -K flag disabled).
    [/quote]
    I
    think you don't understand the difference between AFP and NFS: With NFS
    you setup a static mount (or, at least a mount independent from any
    user login) to the server and that's it. So nothing has to be done at
    login time., just point to the right directory, and the user has all
    his data ready waiting for him. For this reason NFS also works well
    with ssh logins, because multiple users can have directories
    accessible to them mounted simultaneously and in the same place. With
    AFP you have to perform the mount operation at login time. So there is
    a very very big difference in using the two.

    [/quote]
    Yes, and once again you are missing my point: there is nothing known in advance to be not working in Kerberized AFP support, while the -K flag in nfs_mount is known to be disabled.

    Although actually reading Apple's documentation (and don't ask me to look this up again; it's in one of the OS X Server How-To docs) it would appear that the clients aren't just mounting the thing at login, but unmounting it (as guest) and REmounting it (as the user), which is even worse than just mounting it at login.
    [quote user="caleridas"][quote user="The Vicar"] Apple's documentation actually says you can use Kerberized AFP for network home directories, so what you are trying to do should be possible.[/quote]
    Actually I fail to find a website that explicitly states that. The documentation states that you can use AFP with Kerberos; and it states that you can use AFP for home directories. The combination of both seems implied, but maybe it is wrong.[/quote]
    No, I found several documents from Apple which specifically stated you could use Kerberized AFP home directories via Mac OS X Server. (Once again, look it up. I'm too tired,) Besides, abstraction should prevent this from being an issue in any case. The filesystem doesn't know what kind of authentication you used for your AFP login, it only cares that it's AFP.)
    [quote user="caleridas"]
    [quote user="The Vicar"]The solution must be some difference between what your systems are doing and what systems using Apple's built-in system do. So, what could that be?[/quote]
    Errm... maybe they just silently use plaintext authentication with no one noticing?[/quote]
    I really, really suspect that isn't true.
    [quote user="caleridas"]You see, one OS X administrator I know of was quite surprised when I showed him the network protocols dumps...[/quote]
    Those dumps would be surprising anyway.
    [quote user="caleridas"]
    [quote user="The Vicar"]There is nothing special about a "home directory" connection in AFP (as opposed to other connections). So if your clients can get a Kerberized AFP connection at all from netatalk, the problem does not lie with netatalk.[/quote]
    There are several things special about home directory mounts (not from the server or protocol point of view, but by how it is facilitated on the client). Since the home directory can be mounted anywhere, and users are normally not permitted to mount arbirtrary shares anywhere beside the Network folder, the operation is performed with a different privilege set. I *think* mnthome reroutes the mount request through the automounter; this implies that mnthome has to pass any authentication tokens through to the automounter which then performs the operation "on behalf of" the user (but not in the user's privilege context).[/quote]
    I said "connection", didn't I? Not the mount itself, the connection between the client and the server. There is no part of the AFP protocol which says "this is a home directory mount".
    [quote user="caleridas"]
    I hope you understand that some extra work has to be performed for home directories and that there *is* in fact a difference.[/quote]
    I do. I hope you don't display this genius for misunderstanding people all the time.
    [quote user="caleridas"]Ouch... LDAP is *not* an authentication service (though it can be (ab)used as one, it is a terribly bad idea). It is a directory service. As for the fields in the LDAP directory... they are documented reasonably well on the Apple website. Besides the fact that I *know* what OS X server puts in there, and it is not much different. Im am pretty certain that there is nothing to configure on the server side.[/quote]
    Okay, I confess to having typed the wrong word here. (What do you expect? I check this board late at night.) The point is, though, I still would be willing to bet there's a different LDAP key that Mac OS X Server passes along, possibly on the fly, for its Kerberized AFP logins.
    [quote user="caleridas"]
    [quote user="The Vicar"]
    3) Both

    In all three cases, whatever is necessary would be something that someone with Mac OS X Server would be able to detect for you if you asked them. That's why I'm suggesting you contact this guy at Indiana University, since he obviously has studied Mac OS X Server with respect to Kerberos; he probably already knows what you need to do.

    [/quote]

    I admit that there may be something missing. But what really annoyed me was the sorry state of error reporting and logging. Basically when anything goes wrong you never get to see a message *that* something went wrong, *where* it went wrong (not even to mention *why* it went wrong); getting the whole system into at least the state it is currently in required a tremendous amount of server-side diagnosing of client-side problems. I am not used to this hackish working-style.[/quote]

    Well, as I said: Apple expects you to fork over the cash for (and then use) a copy of their Server product, which I suspect would do this specific task (Kerberized AFP home directories) without a problem. The minute your setup includes a non-Apple AFP server (netatalk) and a non-Apple Kerberos keyserver, all bets are off, and they aren't going to give you any help fixing things. I'm not defending this, it's just the reason for why things work this way. Someone at Apple saw what Microsoft does with the different versions of Windows ("Server" versus "Professional" versus "Home", etc.) and got jealous.

    [quote user="caleridas"]

    And I am also not used to the kind of documentation I could find... lots of extremely detailed step-by-step instruction for simple things (which I could have figured out myself, thank you), but no concise explanation of concepts that would let me logically deduce more complicated things.

    [/quote]

    Once again: pay for it, and you get good stuff. Don't pay and you get the minimally helpful stuff. It's the same with developer documentation: the stuff on their website often makes a jump from the so-broadly-conceptual-it-doesn't-help to the so-specific-you-can't-figure-out-how-things-work-together, although either I'm getting more used to decoding the stuff they provide for free or else they're improving. Back when I had access to the paid developer stuff, at least, it was pretty striking how much better the documents you got on the CDs were than what you could get off their website.

    It's kind of depressing. But, on the other hand, they're generally pretty consistent. (So, as far as I can tell, is Microsoft.) Open source, you may or may not be able to find documentation at all. (I just spent some time trying to figure out which Linux distro to download for an old machine that wasn't too good even in its prime... it's amazing how few of them will tell you up front how much RAM is needed to run comfortably. Specific hardware compatability, yes, RAM requirements no.)

    I see from your other post, by the way, that 10.4.8 solved your problem. Well, good! If you ever figure out why, please come back and tell me. For what it's worth, the only thing which is explicitly listed on the 10.4.8 description page as a change in the update has nothing to do with extra LDAP keys; in short, if the initial exchange for the AFP connection is slower than a certain threshold, then Mac OS X will break the data into smaller chunks than otherwise; 10.4.8 increased the minimum size of these chunks by a factor of 4. But it still is (probably) not enough to explain the lack of Kerberized authentication because the minimum size was already 8K.

    On the other hand, the 10.4.8 update also rolls in recent security updates, and the most recent one made some changes to LoginWindow, including fixing two Kerberos-related bugs. (Although neither one is the one you've had, it's still modifications to security w.r.t. Kerberos.)

    And now, off to bed, before my eyes involuntarily close and lock for the night.

Log in to reply