Python is one of the most widely used programming languages today. This section is a large archive of tutorials, based on Python programming language. We cover the basic and advanced technical aspects through coding examples and snippets.
For an in-depth understanding of the above concepts, check out the sections below.
A detailed guide on how to use Python library joblib for parallel computing in Python. Tutorial explains how to submit tasks to joblib pool and then retrieve results. It even explains how to use various parallel computing backend like loky, threading, multiprocessing, dask, etc.
An in-depth guide to configure loggers from dictionary and config files. Python module "logging.config" provides us with various methods to create loggers from configuration details available through dictionary and config files. The tutorial explains methods (dictConfig() & fileConfig()) of module with simple examples.
A simple guide to profile Python code using libraries cProfile and profile. Both are available through standard python installation. They let us measure execution time of function calls made. Tutorial explains how to use libraries to profile code in Python script/program, from command line/shell, and in Jupyter Notebook as well.
A detailed guide on how to use logging module of Python to log events. Tutorial provides an explanation of how to log events of different types (warning, error, info, debug, etc), format error messages, log messages to files, filter log messages, etc. It is useful for basic logging as well as advanced logging for application with many sub-modules.
A comprehensive guide on how to use Python module tracemalloc to profile memory usage by Python code/script/program. Introduced in v3.4, it is a memory profiler that comes with default Python installation. Tutorial explains whole API of tracemalloc with simple examples.
A brief guide to using magic commands available in Jupyter notebooks. Magic commands are special commands starting with a single or double percent sign (%) that let us perform tasks that otherwise need a shell or another tool. Both line magic commands (%) and cell magic commands (%%) are covered in tutorial.
A simple guide on how to display rich media contents (rich outputs) like audio, video, image, animation, JSON, Latex, File links, Code, HTML, etc in the Jupyter notebooks. Python module IPython provides a list of methods (starting with "display_*()") that let us display contents of these types in Notebooks.
A simple guide on how to profile your python code/script/program using line_profiler library that provides run time of code line by line. It explains how to profile whole script from command line ("kernprof") and individual parts of code ("LineProfiler") as well. The library also provides magic command (""%lprun") for usage in Jupyter notebooks.
A comprehensive guide on how to use Python library "imaplib" to manage mailboxes (Gmail, Yahoo, etc). Tutorial covers various operations with mailbox like login/logout, list/create/rename/delete directories, search emails, read emails, copy emails, delete emails, mark emails as read/unread, flag emails as important, etc. It uses IMAP4 protocol behind the scene to communicate with the mailbox server.
A detailed guide on how to use Python library "memory_profiler" to profile memory usage by Python code/script/program and processes. Tutorial covers various ways of profiling with "memory_profiler" like "@profile decorator", "mprof shell command", "memory_usage() function", etc. It even covers how to use "memory_profiler" in Jupyter notebook using "%mprun" and "%memit" magic commands.
Parallel Computing is a type of computation where tasks are assigned to individual processes for completion. These processes can be running on a single computer or cluster of computers. Parallel Computing makes multi-tasking super fast.
Python provides different libraries (joblib, dask, ipyparallel, etc) for performing parallel computing.
Concurrent computing is a type of computing where multiple tasks are executed concurrently. Concurrent programming is a type of programming where we divide a big task into small tasks and execute these tasks in parallel. These tasks can be executed in parallel using threads or processes.
Python provides various libraries (threading, multiprocessing, concurrent.futures, asyncio, etc) to create concurrent code.
Python is an interpreter-based language and hence is slow compared to compiler-based languages (C / C++ / Java, etc). But we can make it super fast (Almost as fast as C++).
Python has a library named 'Numba' that can help us with it. Numba is a JIT Compiler (Just-In-Time) of Python. It converts Python code to faster machine-level code using 'LLVM' compiler.
To convert python code to low-level machine code, it provides us with various decorators (@jit, @njit, @vectorize, @guvectorize, @stencil, etc.). We can decorate our Python functions using these decorators to speed up our Python functions.
Numba even let us parallelize compiled code on multiple CPUs / GPUs for even faster completion.
Python provides various libraries to work with emails. Libraries to send emails, manage mailboxes and represent emails are different based on HTTP protocol.
To send emails, Python provides a library named 'smtplib' (based on SMTP protocol).
To manage mailboxes (read emails, flag emails, move emails, create folder / directory, delete email / directory, etc), Python offers a library named 'imaplib' (based on IMAP protocol).
To represent emails, Python has a library named 'email'. To determine MIME Type of attachment file, Python offers a library named 'mimetypes'.
Profiling is the process of recording time taken by Python program / code / script / process. It measures the time complexity of a program.
Apart from time, we can also record memory usage (space complexity) by our code / program. The process of recording memory usage is referred to as 'memory profiling'. Profiling can help us make better decisions regarding resource allocation as well it presents many opportunities to optimize existing code for maximum resource utilization.
Python has many libraries (cProfile, profile, line_profiler, memory_profiler, tracemalloc, pprofile, scalene, yappi, guppy, py-spy, pyinstrument, etc.) to profile time and memory usage by our Python code.