How are passwords stored in Linux (Understanding hashing with shadow utils)
A user account with a corresponding password for that account, is the primary mechanism that can be used for getting access to a Linux machine. Its very much logical to think that the passwords of all the user's in a system must first be saved in some kind of a file or a database, so that it can be verified during a user login attempt.
And you do not require the skill set and expertise of a computer security scientist to think rationally that if you get hold of that database or file, which stores all the passwords, you can easily get access to the machine.
Some people will argue at this point that the database, or the file that contains the passwords, are in an encoded format (hash value), and its not possible to know the real password, from the encoded password hash. However its possible to detect the real password from the encoded ones, by doing a dictionary attack against the encoded value. Let's get back to some basics before going ahead.
Hashing is a mathematical method to produce a fixed length encoded string for any given string. The main strength of the hashing algorithm is the fact that, you cannot detect the original string from the encoded string. In other words you can call it one way hashing. hashing algorithm's are not only used for storing passwords but also used for data integrity check. You will get a unique fixed length encoded string for any data you give, and that encoded string will be unique to that particular data.
In short no two data can have the same hash (encoded string). Changing a single character of the data will produce a completely different encoded hash(Because the data is different). The strength of hashing is used in almost all communication protocol's in some or the other way to check and confirm the originality and integrity of the data(because, if you alter the data the hash will change).
But what if an attacker gets hold of the encrypted hash and goes on trying different combination of words to produce the exact same hash. With the advanced computing available these days, the attacker can go through millions of combinations in hours. This risk was also applicable to the way passwords were stored in UNIX/Linux system's. Although the passwords were encoded, if an attacker get's hold of that password file, then he can attempt to break the password.
From the initial days of Unix, passwords were stored in a file called as /etc/password. And a major loophole in this single file, is that the file is world readable. Which means any user in the system can read that file. This was done purposely because the file contains critical user related information other than passwords, and many applications and system tools require that information to function properly. Lets see the permission of the /etc/password file in linux.
[root@slashroot1 ~]# ll /etc/passwd -rw-r--r-- 1 root root 1875 Dec 14 23:17 /etc/passwd [root@slashroot1 ~]#
You can see that there is an "r"(Which stands for read), in all the three fields for that file. Let's try to change this permission, so that only "root" can read the file.
[root@slashroot1 ~]# chmod 600 /etc/passwd [root@slashroot1 ~]# ll /etc/passwd -rw------- 1 root root 1875 Dec 14 23:17 /etc/passwd [root@slashroot1 ~]#
Now let's see what's the effect of this change on the system...Let's try to become a normal user, and see what happens.
[root@slashroot1 ~]# su - sarath su: warning: cannot change directory to /home/sarath: Permission denied id: cannot find name for user ID 500 -bash: /home/sarath/.bash_profile: Permission denied -bash-3.2$
oh did you see that...You cant even change the user. Even if you change the user, you will not get the user name, home directory, custom shell etc (Because all these details are stored in /etc/passwd). Due to this reason the file /etc/passwd, needs to be kept world readable. But we cannot keep passwords in a file that's world readable(because of the risk involved, even though its encoded in a one way hash algorithm)
Hence there arises a need to separate passwords from this file and keep it in a file, that's only accessible by root. The solution to this problem is implemented in the form of a package in Linux called "shadow-utils".
What is shadow-utils package in Linux?
Shadow utils is a package in Linux that's installed by default in most of the distributions, used for separating passwords from /etc/passwd. After implementing shadow-utils, passwords are now saved in /etc/shadow file in Linux. This /etc/shadow file is only accessible by root. Let's see the contents of the /etc/shadow file, and also its permission.
[root@slashroot1 ~]# ll /etc/shadow -r-------- 1 root root 1140 Dec 14 23:17 /etc/shadow [root@slashroot1 ~]#
You can see that unlike the /etc/passwd file the /etc/shadow file only has the "r" (read) permission set for root user. Which means no other user has access to this file. Let's see what's the content of this file.
[root@slashroot1 ~]# cat /etc/shadow root:$1$Etg2ExUZ$F9NTP7omafhKIlqaBMqng1:15651:0:99999:7:::
Let's understand each and every field of that output, that are separated by a ":".
1. The first field is self explanatory, its the USERNAME
2. The second field is the encoded password (Which is a one way hash..we will be discussing this in detail)
3. The third field is the day's since the UNIX time that password was changed.
Refer: What is UNIX time?
4. This field specifies the number of days, that are required between password changes.
5.No of days after which its necessary to change the password.
6.This is the number of days before the required password change, the user gets a warning
7.If the password has expired, after this number of days the account will be disabled
8.No of days from the Unix Time, the account is disabled
9. This field is not used yet...
Now you will be confused, that why does the /etc/shadow, file contains these many information's rather than only the encoded password. This is because shadow-util's package provides some more advanced feature's along with storing encoded passwords in /etc/shadow. The above mentioned fields of /etc/shadow, file tell's those added feature's to a certain extent like age of the passwords and its expiry, and also below mentioned feature's.
- Default parametres for user account creation (/etc/login.defs)
- Tools to modify user accounts and groups
- Enforcing strict password selection
How is an encoded password stored in /etc/shadow file, and how does the system verify a user typed password during login?
For understanding how this entire thing works, let's take the case of the previously shown example entry for root user, from /etc/shadow file.
[root@slashroot1 ~]# cat /etc/shadow root:$1$Etg2ExUZ$F9NTP7omafhKIlqaBMqng1:15651:0:99999:7:::
From the above shown example entry, our topic of interest is the second field(the field with the encoded hash of the password).
The above shown encoded hash value can be further classified into three different fields as below.
1. The first field is a numerical number that tell's you the hashing algorithm that's being used.
- $1 = MD5 hashing algorithm.
- $2 =Blowfish Algorithm is in use.
- $2a=eksblowfish Algorithm
- $5 =SHA-256 Algorithm
- $6 =SHA-512 Algorithm
2. The second field is the salt value
Salt value is nothing but a random data that's generated to combine with the original password, inorder to increase the strength of the hash..
3.The last field is the hash value of salt+user password (we will be discussing this shortly).
So in our example entry of root, as shown below,
The above shown encoded password is using MD5 hashing algorithm (because the of $1$)
Salt value is Etg2ExUZ (the content between the second and third $ sign)
And the hash value of "PASSWORD + SALT". Let' s reproduce the same output by providing the salt value of Etg2ExUZ and the original password.
[root@slashroot1 ~]# openssl passwd -1 -salt Etg2ExUZ redhat $1$Etg2ExUZ$F9NTP7omafhKIlqaBMqng1 [root@slashroot1 ~]#
With the help of the above openssl command, you can see that the encoded entry can only be reproduced with the exact same salt value (which is always randomly selected by the password program).
This is what is done by the login program when you enter the password, it uses the salt value and your entered password to create an encoded string. If that encoded string matches the encoded string from the shadow file, then the user login is considered as successful.
Changing the salt will change the entry in shadow file. -1 option used in the above command, tell's which hashing algorithm to use( 1 indicates md5 algorithm).
What will happen if there is no salt value at all?
- Salt value is a major component that strengthens the way a linux system stores password. Imagine that there is no salt value applied before storing passwords in linux. As we have discussed in the beginning of this article, a dictionary attack with common dictionary words will become much more easier to do.
By using the salt value(which is randomly generated while generating passwords), an attacker needs to go throgh different combinations of salt values as well as password string's to guess what the original password is.
- An attacker cannot easily guess that two user's are using same passwords. Because even if the attacker has somehow gained access to the shadow file, he cannot say looking at two encoded passwords, that they are using the same password. This is because both of them will be having different salt values.
How to Display hashing Algorthm used in your Linux Machine?
[root@localhost ~]# authconfig --test|grep hashing password hashing algorithm is md5
Above command clearly shows that, at present the algorithm used by your Linux machine is md5(Which will be used for all the user's by default).
How will you configure your Linux machine to use SHA-512 hashing algorithm?
[root@localhost ~]# authconfig --passalgo=sha512 --update
Please note the fact that unless you change the password of your users, SHA-512 hashing algorithm will not be updated for already existing users.
You can also force your Linux users to change their passwords during the next login by using chage command(which is also provided by the shadow-utils package).
I am doing it for the user tiwary, so that user will need to compulsorily, change the password on next login.
[root@localhost ~]# chage -d 0 tiwary
Above command will force user tiwary to change his password at next Login.
Now when you see /etc/shadow file you can see that the user tiwary, has a different algorithm than all other user's after the password change.
[root@localhost ~]# cat /etc/shadow root:$1$flVALfyK$kJfaoYnsAm7/plT3.PCmJ/:15816:0:99999:7::: satish::15804:0:99999:7::: slashroot:$1$MIyV9col$Up9YON8Z.TI1x37xgFvuO0:15804:0:99999:7::: u1::15813:0:99999:7::: himanshu:$1$0Iwvz7CA$QOJLfOSJZuSLC19LSFxt1.:15810:0:99999:7 tiwary:$6$QXBJkK8N$owDISwfo1wMH1BqoL9Rx/RD49jXpS/EZS8NbIgAUGLtZbkA1YDfZoM1GbeIlDp4p99ugzBMcijNYrxy2rrRdY0:15819:0:99999:7:::
In the above shown output $6$ stands for sha512 algorithm.
How to generate a shadow style password hash?
[root@localhost ~]# openssl passwd -1 redhat123 $1$jp5rCMS4$mhvf4utonDubW5M00z0Ow0
In this case salt value is the eight characters between the 2nd and 3rd $ sign. i.e
You can paste the above output into your shadow file for a particular user and then that user can easily login with the password "redhat123".