hashlib - Compute Secure Hashes (Message Digests) in Python¶
Computing hash is the process of applying a deterministic mathematical function to a message of any size and transform it to a bit array of fixed size. The function used to compute a hash is a one-way function hence we can not revert the transformed message to get the original message. The only way to get the original message is by trying brute force search on messages to generate a hash that matches with the hash at hand. The hash functions commonly used in cryptography are referred to as secure hash functions. Some people refer to hash as message digest as well. The secure hash has many applications like indexing data in the hashtable, digital signatures, detect duplicates, uniquely identify files, use as checksums to check for corruption, etc.
Python provides us with a module named hashlib which provides an implementation of the majority of commonly used hash functions. As a part of this tutorial, we'll explain how we can use the API of hashlib to generate hashes of messages with simple and easy-to-understand examples.
The hashlib provides a list of algorithms which will be available on all platform and some of the algorithms availability will be depended on OpenSSL library that Python uses. The hashlib provides a constructor for algorithms that are available on all platforms as well. The algorithms which have a dependency on the OpenSSL version are available through a method named new().
The hashlib provides two important attributes which let us know the list of secure hashing algorithms available on a particular system.
- algorithms_available - It returns a list of all possible algorithms available on the platform. It'll include available through the OpenSSL library installed on the system.
- algorithms_guaranteed - It returns a list of algorithm which are implemented in this library and are guaranteed to work. The hashlib provides constructors for each of these algorithms.
Below we have printed a list of algorithms available on the system on which this tutorial was run.
import hashlib
print("List of Available Algorithms to Construct Secure Hash/Message Digest : {}".format(hashlib.algorithms_available))
print("\nList of Algorithms Guaranteed to Work : {}".format(hashlib.algorithms_guaranteed))
print("\nList of Algorithms that May Work : {}".format(hashlib.algorithms_available.difference(hashlib.algorithms_guaranteed)))
Example 1: SHA1¶
As a part of our first example, we'll explain how we can compute the secure hash of a given string using SHA1 hashing algorithm. The SHA1 algorithm generates 20 bytes hash of the given input message by working on a block of 64 bytes of data at a time.
- sha1(message_bytes=None) - This constructor creates an instance of sha1 with the initial message given as bytes. It can be then used to generate a message digest. We can only create an instance of sha1 as well without any initial message. We can add messages using a call to update() method.
Important Methods of sha1 Instance¶
update(message_bytes) - This method adds messages given as input to already an existing message. We can call this method more than once and it'll keep on adding up messages.
digest() - It returns message digest of the data in bytes format. The size of the output is 20 bytes.
- hexdigest() - It returns message digest of data as hexadecimal digits. The hexadecimal digest will be twice the size of bytes digest because one byte can represent two hexadecimal digits. The size of the output is 40 hexadecimal digits.
Important Attributes of sha1 Instance¶
- digest_size - It returns an integer representing the number of bytes of secure hash that the algorithm will generate.
- block_size - It returns an integer value representing the block size of the algorithm. The secure hash algorithms divide data into blocks where each block contains a number of bytes of data on which algorithm will work.
Our code for this example starts by creating an instance of sha1 with the input string defined earlier. It gives input in bytes format by calling encode() method on the input string. It then generates messages digests and prints them. The second part of the code again creates an instance of sha1 but without any initial message. It uses update() method to update the message of the algorithm. It then prints the message digest. At last, we print the digest size and block size of the algorithm.
Please make a NOTE that we'll be using secure hash and message digests terms interchangeably but both mean the same thing.
import hashlib
message = "Welcome to CoderzColumn."
########## 1 ##################
sha_1 = hashlib.sha1(message.encode())
message_digest1 = sha_1.digest()
print("Message Digest 1 : {}".format(message_digest1))
########## 2 ##################
sha_2 = hashlib.sha1()
sha_2.update(bytes(message, encoding="utf-8"))
message_digest2 = sha_2.digest()
print("Message Digest 2 : {}".format(message_digest2))
print("\nMessage Digest Size for 1 : {} and 2 : {}".format(sha_1.digest_size, sha_2.digest_size,))
print("Message Block Size for 1 : {} and 2 : {}".format(sha_1.block_size, sha_2.block_size,))
Example 2: SHA1 Continued ...¶
As a part of our second example, we are continuing with an explanation of using SHA1 algorithm. We are using our algorithm for this part to explain how we can call update() method more than one time to give a message in parts and it'll give the same results if data is given in one go.
Our code first two parts of the code are the same as our previous example. Our third part of the code first creates an instance of sha1 and then calls update() method two times giving half of the original message each time. We then print message digest.
When we run the below code, we can notice that all message digests are equal.
Please make a NOTE that all secure hash algorithms available through hashlib require input in bytes format and will return output message digest also in bytes format.
import hashlib
message = "Welcome to CoderzColumn."
########## 1 ##################
sha_1 = hashlib.sha1(message.encode())
message_digest1 = sha_1.digest()
print("Message Digest 1 : {}".format(message_digest1))
########## 2 ##################
sha_2 = hashlib.sha1()
sha_2.update(bytes(message, encoding="utf-8"))
message_digest2 = sha_2.digest()
print("Message Digest 2 : {}".format(message_digest2))
########## 3 ##################
sha_3 = hashlib.sha1()
sha_3.update(bytes("Welcome to ", encoding="utf-8"))
sha_3.update(bytes("CoderzColumn.", encoding="utf-8"))
message_digest3 = sha_3.digest()
print("Message Digest 3 : {}".format(message_digest3))
print("\nMessage Digest Size for 1 : {}, 2 : {} and 3 : {}".format(sha_1.digest_size, sha_2.digest_size,sha_3.digest_size,))
print("Message Block Size for 1 : {}, 2 : {} and 3 : {}".format(sha_1.block_size, sha_2.block_size,sha_3.block_size,))
Example 3: SHA1 - Hex Message Digest¶
As a part of our third example, we are explaining how we can generate hex message digest using SHA1 algorithm.
Our code for this example is exactly the same as our previous example with the only change that we are using hexdigest() method to calculate digest instead of digest() method.
import hashlib
message = "Welcome to CoderzColumn."
########## 1 ##################
sha_1 = hashlib.sha1(message.encode())
hex_message_digest1 = sha_1.hexdigest()
print("Hex Message Digest 1 : {}".format(hex_message_digest1))
########## 2 ##################
sha_2 = hashlib.sha1()
sha_2.update(bytes(message, encoding="utf-8"))
hex_message_digest2 = sha_2.hexdigest()
print("Hex Message Digest 2 : {}".format(hex_message_digest2))
########## 3 ##################
sha_3 = hashlib.sha1()
sha_3.update(bytes("Welcome to ", encoding="utf-8"))
sha_3.update(bytes("CoderzColumn.", encoding="utf-8"))
hex_message_digest3 = sha_3.hexdigest()
print("Hex Message Digest 3 : {}".format(hex_message_digest3))
print("\nHex Message Digest Size for 1 : {}, 2 : {} and 3 : {}".format(sha_1.digest_size, sha_2.digest_size,sha_3.digest_size,))
print("Hex Message Block Size for 1 : {}, 2 : {} and 3 : {}".format(sha_1.block_size, sha_2.block_size,sha_3.block_size,))
Example 4: Use of new() Method to Create Hash Algorithm Instance¶
As a part of our fourth example, we are demonstrating how we can use new() method to create an instance of a hashing algorithm using new() method.
- new(name,[data]) - This constructor creates an instance of algorithm given as input by the name parameter. It provides access to algorithms for which there is no direct constructor available in hashlib and is dependent on the OpenSSL library.
Our code for this example, generates an instance of sha1 and md5 algorithms using new() method. It then generates message digests and prints them. We also print digest size and block size at the end for each algorithm. The MD5 algorithm generates a digest of size 16 bytes and works on blocks of data of size 64 bytes.
import hashlib
message = "Welcome to CoderzColumn."
########## 1 ##################
sha_1 = hashlib.sha1(message.encode())
sha_1_new = hashlib.new(sha_1.name, data=message.encode())
message_digest1 = sha_1_new.digest()
print("Message Digest 1 : {}".format(message_digest1))
########## 2 ##################
sha_2 = hashlib.new("sha1")
sha_2.update(bytes(message, encoding="utf-8"))
message_digest2 = sha_2.digest()
print("Message Digest 2 : {}".format(message_digest2))
########## 2 ##################
md5 = hashlib.new("md5")
md5.update(bytes(message, encoding="utf-8"))
message_digest3 = md5.digest()
print("Message Digest 3 : {}".format(message_digest3))
print("\nMessage Digest Size for 1 : {}, 2 : {}, 3 : {}".format(sha_1.digest_size, sha_2.digest_size, md5.digest_size))
print("Message Block Size for 1 : {}, 2 : {}, 3 : {}".format(sha_1.block_size, sha_2.block_size, md5.block_size))
Example 5: SHAKE128¶
As a part of our fifth example, we are demonstrating the usage of SHAKE128 secure hashing algorithm. It let us compute the message digest of variable length providing security of 128 bits. The SHAKE_128 works on 128 bytes block of data at a time.
- shake_128(message_bytes=None) - It creates and instance of shake_128 which can be used to compute digest.
Important Methods of shake_128 Instance¶
- digest(digest_size_in_bytes) - It returns an message digest of given byte size.
- hexdigest(digest_size_in_bytes) - It returns an hex message digest of given byte size. The length of the hex digest will be double the size normal digest because one byte can represent two hexadecimal digits. The number of bytes used will be the same as a normal digest.
Our code for this example creates and instances of shake_128 and generates normal and hex message digests of size 15 bytes.
import hashlib
message = "Welcome to CoderzColumn."
########## 1 ##################
shake_128_1 = hashlib.shake_128(message.encode())
message_digest1 = shake_128_1.digest(15)
print("Message Digest 1 : {}".format(message_digest1))
hex_message_digest1 = shake_128_1.hexdigest(15)
print("Hex Message Digest 1 : {}".format(hex_message_digest1))
########## 2 ##################
shake_128_2 = hashlib.new("shake_128")
shake_128_2.update(bytes(message, encoding="utf-8"))
message_digest2 = shake_128_2.digest(15)
print("\nMessage Digest 2 : {}".format(message_digest2))
hex_message_digest2 = shake_128_2.hexdigest(15)
print("Hex Message Digest 2 : {}".format(hex_message_digest2))
print("\nMessage Digest Size for 1 : {}, 2 : {}".format(shake_128_1.digest_size, shake_128_2.digest_size))
print("Message Block Size for 1 : {}, 2 : {}".format(shake_128_1.block_size, shake_128_2.block_size))
Example 6: SHAKE256¶
As a part of our sixth example, we have explained how we can use SHAKE256 algorithm to generate secure hashes. It works like SHAKE128 but provides the security of 256 bits. All the methods of the shake_256 instance are the same as shake_128 instance.
Our code for this example is exactly the same as our previous example with the only change that we are using shake_256 algorithm for this example.
import hashlib
message = "Welcome to CoderzColumn."
########## 1 ##################
shake_256_1 = hashlib.shake_256(message.encode())
message_digest1 = shake_256_1.digest(15)
print("Message Digest 1 : {}".format(message_digest1))
hex_message_digest1 = shake_256_1.hexdigest(15)
print("Hex Message Digest 1 : {}".format(hex_message_digest1))
########## 2 ##################
shake_256_2 = hashlib.new("shake_256")
shake_256_2.update(bytes(message, encoding="utf-8"))
message_digest2 = shake_256_2.digest(15)
print("\nMessage Digest 2 : {}".format(message_digest2))
hex_message_digest2 = shake_256_2.hexdigest(15)
print("Hex Message Digest 2 : {}".format(hex_message_digest2))
print("\nMessage Digest Size for 1 : {}, 2 : {}".format(shake_128_1.digest_size, shake_128_2.digest_size))
print("Message Block Size for 1 : {}, 2 : {}".format(shake_128_1.block_size, shake_128_2.block_size))
Example 7: Password Hashing using pbkdf2_hmac()¶
As a part of our seventh example, we'll demonstrate how we can use key derivation/key stretching algorithm PKCS#5 for generating a secure hash of passwords.
The reason behind using different functions based on salt input bytes for hashing passwords is to protect them against brute force attacks.
- pbkdf2_hmac(hash_algo, password, salt, iterations,dklen=None) - It takes as input hashing algorithm name, byte sequence to use as a password, byte sequence to use like salt, number of iterations, and returns a hash of input password. If dklen parameter is provided with some number then a digest of that many bytes will be generated else digest of size will be based on hashing algorithm given as input.
Our code for this example generates secure hash of input string using pbkdf2_hmac() function with different parameter settings. All inputs are given as bytes sequence.
import hashlib
password = "CoderzColumn"
salt = "CC"
########## 1 ##################
hashed_pass = hashlib.pbkdf2_hmac("sha256", password=password.encode(),
salt = salt.encode(),
iterations=100000)
print("Password Hash ({}) : {}".format(len(hashed_pass), hashed_pass))
########## 2 ##################
hashed_pass = hashlib.pbkdf2_hmac("md4", password=password.encode(),
salt = salt.encode(),
iterations=100000)
print("\nPassword Hash ({}) : {}".format(len(hashed_pass), hashed_pass))
########## 3 ##################
hashed_pass = hashlib.pbkdf2_hmac("sha256", password=password.encode(),
salt = salt.encode(),
iterations=1000000,
dklen=64)
print("\nPassword Hash ({}) : {}".format(len(hashed_pass), hashed_pass))
Example 8: Password Hashing using scrypt()¶
As a part of our eighth example, we'll explain how we can generate secure has for passwords using key derivation/key stretching algorithms. We are using scrypt() method of hashlib this time.
- scrypt(password,salt,n,r,p,maxmem=0,dklen=64) - It takes as input password and other parameter values and generates hash of password.
- The parameter n is a value representing the CPU/Memory cost factor.
- The parameter r represents the block size of the algorithm.
- The parameter p represents the parallelization factor.
- The parameter maxmem accepts a number that represents an upper bound on memory usage by the algorithm. The default value is 0 hence no limitation.
- The parameter dklen represents the required digest size in bytes. The default is 64 bytes.
Our code for this example generates a secure hash of the password using scrypt() method. It tries different parameter combinations to generate the hash.
import hashlib
password = "CoderzColumn"
salt = "CC"
########## 1 ##################
hashed_pass = hashlib.scrypt(password=password.encode(),
salt = salt.encode(),
n=2,
r=16,
p=1
)
print("Password Hash ({}) : {}".format(len(hashed_pass), hashed_pass))
########## 2 ##################
hashed_pass = hashlib.scrypt(password=password.encode(),
salt = salt.encode(),
n=4,
r=8,
p=1
)
print("\nPassword Hash ({}) : {}".format(len(hashed_pass), hashed_pass))
########## 3 ##################
hashed_pass = hashlib.scrypt(password=password.encode(),
salt = salt.encode(),
n=2,
r=16,
p=1,
dklen=32)
print("\nPassword Hash ({}) : {}".format(len(hashed_pass), hashed_pass))
Example 9: Blake2B¶
As a part of our ninth example, we are demonstrating how we can generate a secure hash of the given message using Blake2B algorithm.
- blake2b(data=b'',digest_size=64,key=b'',salt=b'',person=b'',fanout=1,depth=1,leaf_size=0,node_offset=0,node_depth=0,inner_size=0,last_node=False,usedforsecurity=False) - This constructor creates an instance of blake2b instance which can be used to generate secure hash.
- The data parameter accepts data for which we want to generate a secure hash in bytes format.
- The digest_size accepts number specifying message digest size in bytes.
- The key accepts key in bytes format for keyed hashing.
- The salt accepts a salt string in bytes format for randomizing hashing.
- The person parameter accepts bytes for personalization.
Our code for this example has three parts.
The first part creates an instance of blake2b with just a message as input and all other parameters are left with their default values. We are then generating message digest and hex message digests as well.
Our second part of the code is generating an instance of blake2b with the message, digest size, key, and salt parameters set. We are then generating message digest and hex message digest using this instance.
Our third part of the code is generating an instance of blake2b using new() method described earlier.
At last, we are printing digest size and block size for each instance.
import hashlib
message = "Welcome to CoderzColumn."
########## 1 ##################
blake2b_1 = hashlib.blake2b(message.encode())
message_digest1 = blake2b_1.digest()
print("Message Digest 1 : {}".format(message_digest1))
hex_message_digest1 = blake2b_1.hexdigest()
print("Hex Message Digest 1 : {}".format(hex_message_digest1))
########## 2 ##################
key = bytes("python", encoding="utf-8")
salt = bytes("ml", encoding="utf-8")
print("\nKey Size : {}, Salt Size : {}".format(len(key), len(salt)))
blake2b_2 = hashlib.blake2b(message.encode(), digest_size=32, key=key, salt=salt)
message_digest2 = blake2b_2.digest()
print("\nMessage Digest 2 : {}".format(message_digest2))
hex_message_digest2 = blake2b_2.hexdigest()
print("Hex Message Digest 2 : {}".format(hex_message_digest2))
########## 3 ##################
blake2b_3 = hashlib.new("blake2b", digest_size=32, key=key, salt=salt)
blake2b_3.update(bytes(message, encoding="utf-8"))
message_digest3 = blake2b_3.digest()
print("\nMessage Digest 2 : {}".format(message_digest3))
hex_message_digest2 = blake2b_3.hexdigest()
print("Hex Message Digest 2 : {}".format(message_digest3))
print("\nMessage Digest Size for 1 : {}, 2 : {}, 3 : {}".format(blake2b_1.digest_size, blake2b_2.digest_size, blake2b_3.digest_size))
print("Message Block Size for 1 : {}, 2 : {}, 3 : {}".format(blake2b_1.block_size, blake2b_2.block_size, blake2b_3.block_size))
Example 10: Blake2S¶
As a part of our tenth example, we are demonstrating how we can generate message digest using Blake2S algorithm.
- blake2s(data=b'',digest_size=32,key=b'',salt=b'',person=b'',fanout=1,depth=1,leaf_size=0,node_offset=0,node_depth=0,inner_size=0,last_node=False,usedforsecurity=False) - This constructor creates an instance of blake2s instance which can be used to generate secure hash.
The blake2s algorithm has the same signature as blake2b with only a change in default digest size of 32 instead of 64.
Our code for this example is exactly the same as our previous example with the only minor change that we are using blake2s instance for generating message digest in this example instead.
import hashlib
message = "Welcome to CoderzColumn."
########## 1 ##################
blake2s_1 = hashlib.blake2s(message.encode())
message_digest1 = blake2s_1.digest()
print("Message Digest 1 : {}".format(message_digest1))
hex_message_digest1 = blake2s_1.hexdigest()
print("Hex Message Digest 1 : {}".format(hex_message_digest1))
########## 2 ##################
key = bytes("python", encoding="utf-8")
salt = bytes("ml", encoding="utf-8")
print("\nKey Size : {}, Salt Size : {}".format(len(key), len(salt)))
blake2s_2 = hashlib.blake2s(message.encode(), digest_size=32, key=key, salt=salt)
message_digest2 = blake2s_2.digest()
print("\nMessage Digest 2 : {}".format(message_digest2))
hex_message_digest2 = blake2s_2.hexdigest()
print("Hex Message Digest 2 : {}".format(hex_message_digest2))
########## 3 ##################
blake2s_3 = hashlib.new("blake2s", digest_size=32, key=key, salt=salt)
blake2s_3.update(bytes(message, encoding="utf-8"))
message_digest3 = blake2s_3.digest()
print("\nMessage Digest 2 : {}".format(message_digest3))
hex_message_digest2 = blake2s_3.hexdigest()
print("Hex Message Digest 2 : {}".format(message_digest3))
print("\nMessage Digest Size for 1 : {}, 2 : {}, 3 : {}".format(blake2s_1.digest_size, blake2s_2.digest_size, blake2s_3.digest_size))
print("Message Block Size for 1 : {}, 2 : {}, 3 : {}".format(blake2s_1.block_size, blake2s_2.block_size, blake2s_3.block_size))
Example 11¶
As a part of our eleventh example, we are just looping through all guaranteed available algorithms, create instances of them, and generate normal and hex message digest for the input string. We are printing message digests generated by all algorithms along with their digest size and name as well.
message = "Welcome to CoderzColumn."
for algorithm in hashlib.algorithms_guaranteed:
hashing_algo = hashlib.new(algorithm, message.encode())
print("Algorithm : {}".format(algorithm))
if algorithm in ["shake_128", "shake_256"]:
message_digest1 = hashing_algo.digest(10)
else:
message_digest1 = hashing_algo.digest()
print("Message Digest (Digest Size : {}) : {}".format(len(message_digest1), message_digest1))
if algorithm in ["shake_128", "shake_256"]:
hex_message_digest1 = hashing_algo.hexdigest(10)
else:
hex_message_digest1 = hashing_algo.hexdigest()
print("Hex Message Digest (Digest Size : {}) : {}".format(len(hex_message_digest1),hex_message_digest1))
print("="*80, "\n")
Example 12¶
As a part of our eleventh example, we are just looping through all algorithms which are available through the OpenSSL library, create an instance of them and generate a normal and hex message digest for the input string. We are printing message digests generated by all algorithms along with their digest size and name as well. We have found out a list of OpenSSL algorithms by doing a set difference between algorithms_available and algorithms_guaranteed attributes of hashlib.
message = "Welcome to CoderzColumn."
may_be_algorithms = hashlib.algorithms_available.difference(hashlib.algorithms_guaranteed)
for algorithm in may_be_algorithms:
hashing_algo = hashlib.new(algorithm, message.encode())
print("Algorithm : {}".format(algorithm))
if algorithm in ["shake_128", "shake_256"]:
message_digest1 = hashing_algo.digest(10)
else:
message_digest1 = hashing_algo.digest()
print("Message Digest (Digest Size : {}) : {}".format(len(message_digest1), message_digest1))
if algorithm in ["shake_128", "shake_256"]:
hex_message_digest1 = hashing_algo.hexdigest(10)
else:
hex_message_digest1 = hashing_algo.hexdigest()
print("Hex Message Digest (Digest Size : {}) : {}".format(len(hex_message_digest1),hex_message_digest1))
print("="*80, "\n")
This ends our small tutorial explaining the API of hashlib module to generate the secure hash. Please feel free to let us know your views in the comments section.
References¶
Sunny Solanki
Sunny Solanki
Intro: Software Developer | Youtuber | Bonsai Enthusiast
About: Sunny Solanki holds a bachelor's degree in Information Technology (2006-2010) from L.D. College of Engineering. Post completion of his graduation, he has 8.5+ years of experience (2011-2019) in the IT Industry (TCS). His IT experience involves working on Python & Java Projects with US/Canada banking clients. Since 2020, he’s primarily concentrating on growing CoderzColumn.
His main areas of interest are AI, Machine Learning, Data Visualization, and Concurrent Programming. He has good hands-on with Python and its ecosystem libraries.
Apart from his tech life, he prefers reading biographies and autobiographies. And yes, he spends his leisure time taking care of his plants and a few pre-Bonsai trees.
Contact: sunny.2309@yahoo.in
![YouTube Subscribe]()
Comfortable Learning through Video Tutorials?
If you are more comfortable learning through video tutorials then we would recommend that you subscribe to our YouTube channel.
![Need Help]()
Stuck Somewhere? Need Help with Coding? Have Doubts About the Topic/Code?
Stuck Somewhere? Need Help with Coding? Have Doubts About the Topic/Code?When going through coding examples, it's quite common to have doubts and errors.
If you have doubts about some code examples or are stuck somewhere when trying our code, send us an email at coderzcolumn07@gmail.com. We'll help you or point you in the direction where you can find a solution to your problem.
You can even send us a mail if you are trying something new and need guidance regarding coding. We'll try to respond as soon as possible.
![Share Views]()
Want to Share Your Views? Have Any Suggestions?
Want to Share Your Views? Have Any Suggestions?If you want to
- provide some suggestions on topic
- share your views
- include some details in tutorial
- suggest some new topics on which we should create tutorials/blogs
hashlib, secure-hash, message-digest
Subscribe to Our YouTube Channel
