In this article, I’m going to get right down to the nitty gritty of OS X password implementation on 10.2, 10.3, and 10.4. I assume you have some knowledge of cryptographic hashes and algorithms. I will also assume you have knowledge of Unix password systems, since OS X passwords are (not surprisingly) heavily based on Unix implementations.

The most obvious difference from Unix implementations (which is common to all current versions of OS X) is that the user information is stored not in flat files like /etc/password and friends, but an actual database, called NetInfo. The problem is flat files just don’t scale beyond small systems. The history of NetInfo goes all the way back to NEXSTEP, and is similar in concept to NIS from Sun and the LDAP protocol. All these non-compatible solutions store user information in a database, instead of flat files to improve performance and scalability. These days, LDAP has pretty much beaten out the other two systems in the Unix enterprise, but NetInfo persists on desktop versions of OS X. I presume this is because it works and it’s just not worth changing at this point. NetInfo provides some nice Unix compatibility features. One example is the command to dump the NetInfo user database in standard Unix /etc/password file format:

% nidump passwd .

This is where things start diverging between OS X versions. In OS X 10.2 (and presumably 10.0 and 10.1), the hashed password is stored directly in NetInfo. It also uses the standard Unix DES hash. Since all users have access to the complete NetInfo database, any user on the machine may run the nidump command above and get every user’s hashed DES password. To illustrate this point, let’s say we have a fictitious user with the login of “sjobs” and password of “macintosh”:

% nidump passwd . | grep sjobs
sjobs:3dI880QaIz.Wk:501:501::0:0:sjobs:/Users/jobs:/bin/bash

That’s correct: passwords are not shadowed and it uses the same hashing algorithm used in AT&T Unix Version 7, which was released in 1979. Everything the rest of the Unix world has learned about passwords in the last 25 years has been ignored. 10.2 was released in 2002. Shadow passwords were first introduced by AT&T in 1987. And DES was effectively proven weak in 1997 by RSA’s DES Challenge. DES also truncates passwords to 8 characters, which makes it nearly impossible to use a passphrase, instead of a password. We can verify this password using the openssl command line tool:

% openssl passwd -crypt -salt 3d macintosh
Warning: truncating password to 8 characters
3dI880QaIz.Wk

Fortunately, in 2003, Apple released OS X 10.3 which attempted to fix these issues. First and foremost, passwords are now shadowed and stored completely outside of NetInfo. Now, when a non-privleged user tries to dump the NetInfo user database, they no longer get the password:

% nidump passwd . | grep sjobs
sjobs:********:501:501::0:0:sjobs:/Users/jobs:/bin/bash

Rather than use the standard /etc/shadow file for hashes, they are stored in the /var/db/shadow/hash/ directory. Like /etc/shadow, this is readable only by root. However, each users password is stored in their own file. The file is not the username of the account, but the generateduid NetInfo field. You can view this field from NetInfo manager, or by using niutil:

% niutil -readprop . /users/sjobs generateduid
70902C33-AC79-11DA-AFDF-000A95CD9AF8

The contents of this file is:

% hash_dir=/var/db/shadow/hash
% uid=70902C33-AC79-11DA-AFDF-000A95CD9AF8
% hash_file=$hash_dir/$uid
% sudo more $hash_file
D47F3AF827A48F7DFA4F2C1F12D68CD608460EB13C5CA0C4CA9516712F7FED9501424F955C11F92E\
FEF0B79D7FA3FB6BE56A9F99
% sudo cat $hash_file | wc -c
     104

This is clearly not standard Unix DES anymore. It turns out this string of 104 characters is the password hashed twice: once with SHA1, and again with Windows LM hash, often called LANMAN. The first 64 characters are the LANMAN hash:

% sudo cat $hash_file | cut -c1-64
D47F3AF827A48F7DFA4F2C1F12D68CD608460EB13C5CA0C4CA9516712F7FED95

And the last 40 characters are the SHA1 hash:

% sudo cat $hash_file | cut -c65-104
01424F955C11F92EFEF0B79D7FA3FB6BE56A9F99

There are no command line utilities that calculate the Windows LANMAN password, AFAIK. However, we can verify the SHA1 hash with openssl command line tool:

% echo -n macintosh | openssl dgst -sha1
01424f955c11f92efef0b79d7fa3fb6be56a9f99

The use of both shadowing and SHA1 are a step forward, for the most part. SHA1 is a true hash, so there are no length restrictions. SHA1 is also still considered a fairly secure algorithm that is not easily calculated, even with hardware, unlike DES. Apple, unfortunately, chose not to use a salt. While salts don’t make the password any harder or easier to guess, salts make performing dictionary or brute force attacks on a group of passwords far more expensive. It is a shame that salts are not used.

Lack of salts is not the worst problem in 10.3, though. The fact the passwords are also hashed with LANMAN is a far worse problem. LANMAN is known to be a very weak algorithm. For details, I recommend reading Password Attack Discussion and Benchmarks by Alan Amesbury at the University of Minnesota. In summary, the problems with LANMAN are:

  • Case insensitive<
  • Limited to 14 characters, but effectively limited to 7 characters
  • Computationally cheap
  • No salt<

Alan claims it takes about 17 days to brute force guess all possible LANMAN hashes on a current day PC. That’s pathetically weak.

So why does Apple use LANMAN? For interoperability with Windows, of course. Storing a Windows compatible password allows you to mount your home directory from a Windows machine over the network. Unfortunately, OS X 10.3 creates this Windows compatible hash whether or not you have Windows sharing enabled. This behavior completely nullifies the fact the OS X also hashes the password with the stronger SHA1. A smart attacker will just attack the LANMAN password, instead of SHA1. Sure, the case could be wrong, but it doesn’t take very long go through all combinations of letters in upper and lower case.

It is worth noting that even Windows no longer uses LANMAN, in favor of the stronger NTLM. LANMAN was only used to hash passwords up through Windows ME. I don’t know why OS X still uses LANMAN over the stronger NTLM, other than the fact that OS X uses Samba for Windows file sharing. It could be a limitation in Samba or the SMB/CIFS protocol itself. I really don’t know.

All is not bad, though. The current version of OS X is 10.4, and was released in 2005. It turns out the changes made for 10.4 were all good. The passwords are still shadowed in the same file. However, the LANMAN password is only generated if you have Windows sharing enable. In fact, OS X specifically warns you that your password is being stored in a less secure manner, when you try and enable Windows sharing. This is great. But it doesn’t end there: Apple also added salts to the SHA1 hash. The format of the hash file changed, too:

% sudo more $hash_file
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000\
00000000000000000000000000000000000000000000000000000000000000000000000000000000\
000000000E6A48F765D0FFFFF6247FA80D748E615F91DD0C7431E4D9000000000000000000000000\
00000000000000000000000000000000000000000000000000000000000000000000000000000000\
00000000000000000000000000000000000000000000000000000000000000000000000000000000\
00000000000000000000000000000000000000000000000000000000000000000000000000000000\
00000000000000000000000000000000000000000000000000000000000000000000000000000000\
00000000000000000000000000000000000000000000000000000000000000000000000000000000\
00000000000000000000000000000000000000000000000000000000000000000000000000000000\
00000000000000000000000000000000000000000000000000000000000000000000000000000000\
00000000000000000000000000000000000000000000000000000000000000000000000000000000\
00000000000000000000000000000000000000000000000000000000000000000000000000000000\
00000000000000000000000000000000000000000000000000000000000000000000000000000000\
00000000000000000000000000000000000000000000000000000000000000000000000000000000\
00000000000000000000000000000000000000000000000000000000000000000000000000000000\
0000000000000000000000000000000000000000
% sudo cat $hash_file | wc -c
    1240

The salted SHA1 hash starts at character 169, and is 48 characters long:

% salted_hash=`sudo cat $hash_file | cut -c169-216`
% echo $salted_hash
0E6A48F765D0FFFFF6247FA80D748E615F91DD0C7431E4D9

The first 8 characters are the hex value of a 4-byte salt. To verify this password, we first need to convert the salt back into its integer value:

% hex_salt=`echo $salted_hash | cut -c1-8`
% salt=`echo -n $hex_salt | xxd -r -p`

Now, we have to take the SHA1 hash of the salt and the cleartext password:

% sha=`printf "%s%s" $salt macintosh | openssl dgst -sha1`
% printf "%s%s\n" $hex_salt $sha
0E6A48F765d0fffff6247fa80d748e615f91dd0c7431e4d9

As you can see, this is the same hex value as $salted_hash, above. So, what’s with all the extra zeros in the 10.4 shadow file? The first 64 characters are used for the LANMAN password, if Windows sharing is enabled. The next 40 characters are used for the unsalted SHA1 hash, if the account was upgraded from 10.3. I have no idea what’s with all the other zeros. Perhaps it’s for future compatibility in newer versions of OS X.

So how does OS X compare with other Unices? FreeBSD started using an MD5-based hash in 1994, which is now also used by Linux. MD5 is weaker than SHA1, however, the algorithm used by FreeBSD uses multiple MD5 hashes. This makes it computationally more expensive to compute than straight SHA1, and is an advantage against attacks. The king of the hill has to go to OpenBSD, which uses a hash based on the Blowfish block cipher called bcrypt. The benefit of bcypt is best stated on their website:

The most important property of bcrypt (and thus crypt_blowfish) is that it is adaptable to future processor performance improvements, allowing you to arbitrarily increase the processing cost of checking a password while still maintaining compatibility with your older password hashes. Already now bcrypt hashes you would use are several orders of magnitude stronger than traditional Unix DES-based or FreeBSD-style MD5-based hashes.

This is just plain cool. So what’s in store for OS X 10.5, Leopard? Only Apple knows, but I will be interested to see if they make any changes in the password implementation.

See Also: Part 1 of this series.