Updated On : Aug-16,2022 Time Investment : ~30 mins

logging.config - Simple Guide to Configure Loggers from Dictionary and Config Files in Python

Logging is one of the most critical aspects of software development. The developers log events like successful completion, warnings, errors, fatal errors, etc. These logs can help developers in various ways like finding out the root cause in case when some failure happens in the future, looking for slow parts of the code for optimization, etc.

Python provides a module named logging with a vast API that can be used to log events in Python. We have already covered a very detailed tutorial covering the majority of the API of logging module. Please feel free to check it if you are interested in learning about it (link below).

This tutorial will build on that tutorial and explain how we can configure logging module based on the configuration from dictionary and configuration files. It's recommended to have knowledge of the logging module explained in that tutorial to continue with this tutorial.

> What Can You Learn From This Article?

As a part of this tutorial, we have explained how to use 'logging.config' module to configure logging in Python using configuration provided through a dictionary or config file. The 'config' is a sub-module of 'logging' module that let us modify default configuration.

It let us modify configuration in two ways:

  1. Using Dictionary
  2. Using Configuration File

Below, we have listed important sections of tutorial to give an overview of material covered.

Important Sections Of Tutorial

  1. Dictionary Config Examples
    • 1.1 Dictionary Config to Log Messages with Level DEBUG & Above
      • 1.1.1 How to Define Dictionary for Configuration Schema (Important Dictionary Keys)?
    • 1.2 Create Logger using Dictionary Config
    • 1.3 Dictionary Config to Create Multiple Loggers
    • 1.4 Dictionary Config to Direct Logs to File and Console
    • 1.5 Dictionary Config for Hierarchy Of Loggers
    • 1.6 Dictionary Config to Include Filters
    • 1.7 Dictionary Config for Multiple Filters
  2. File Config Examples
    • 2.1 File Config to Log Messages with Level DEBUG & Above
      • 2.1.1 Configuration File Format Explanation
    • 2.2 File Config to Create Logger
    • 2.3 File Config to Create Different Handlers (Direct Logs to File)
    • 2.4. File Config to Create Hierarchy of Loggers
    • 2.5 File Config to Create Multiple Loggers

We'll now start explaining the configuration of logging with examples.

1. Dictionary Config Examples

1. Dictionary Config to Log Messages with Level DEBUG & Above

As a part of our first example, we'll explain how we can create a simple dictionary configuration that will inform the logging module to direct all logging messages with level DEBUG and above to the standard output stream.


  • dictConfig(config_dictionary) - This method takes as input dictionary which has configuration details about logging and configures logging module based on it.

Our code for this example starts by creating a dictionary that has logging configuration details like root logger, log level, handlers, and formatters. The dictionary has set the log level to DEBUG hence log messages at level DEBUG and above will be logged. We have included StreamHandler handler for handling log messages and directing them to standard output/console. We have also included a formatter in the dictionary which will format log messages to include details like time, log level, module name, function name, line number, process details, and thread details. We have then configured logging by giving this dictionary as input to dictConfig() method.

We have defined a method named addition which takes as input two parameters, performs addition on them, and returns results. It checks data types of input arguments and logs warning messages if not proper data types as given as input. It also logs messages based on the successful completion of the addition. It also logs an error message if there is any failure.

The main part of our code calls addition method with different arguments and prints the result. We are also logging the result of the addition.

When we run the below script, we can notice from the output that it has included all log messages at level DEBUG and above. By default logging module only includes logging messages at level WARNING and above which seems to have been overridden with this configuration.

logging_config_ex_1.py

import logging
from logging import config

log_config = {
    "version":1,
    "root":{
        "handlers" : ["console"],
        "level": "DEBUG"
    },
    "handlers":{
        "console":{
            "formatter": "std_out",
            "class": "logging.StreamHandler",
            "level": "DEBUG"
        }
    },
    "formatters":{
        "std_out": {
            "format": "%(asctime)s : %(levelname)s : %(module)s : %(funcName)s : %(lineno)d : (Process Details : (%(process)d, %(processName)s), Thread Details : (%(thread)d, %(threadName)s))\nLog : %(message)s",
            "datefmt":"%d-%m-%Y %I:%M:%S"
        }
    },
}

config.dictConfig(log_config)

def addition(a, b):
    logging.debug("Inside Addition Function")
    if isinstance(a, str) and a.isdigit():
        logging.warning("Warning : Parameter A is passed as String. Future versions won't support it.")

    if isinstance(b, str) and b.isdigit():
        logging.warning("Warning : Parameter B is passed as String. Future versions won't support it.")

    try:
        result = float(a) + float(b)
        logging.info("Addition Function Completed Successfully")
        return result
    except Exception as e:
        logging.error("Error Type : {}, Error Message : {}".format(type(e).__name__, e))
        return None


if __name__ == "__main__":
    #logging.info("Current Log Level : {}\n".format(logging.getLevel()))

    result = addition(10,20)
    logging.info("Addition of {} & {} is : {}\n".format(10,20, result))

    result = addition("20",20)
    logging.info("Addition of {} & {} is : {}\n".format("'20'",20, result))

    result = addition("A",20)
    logging.info("Addition of {} & {} is : {}".format("A",20, result))

OUTPUT

16-03-2021 05:01:13 : DEBUG : logging_config_ex_1 : addition : 28 : (Process Details : (23515, MainProcess), Thread Details : (139781264041792, MainThread))
Log : Inside Addition Function
16-03-2021 05:01:13 : INFO : logging_config_ex_1 : addition : 37 : (Process Details : (23515, MainProcess), Thread Details : (139781264041792, MainThread))
Log : Addition Function Completed Successfully
16-03-2021 05:01:13 : INFO : logging_config_ex_1 : <module> : 48 : (Process Details : (23515, MainProcess), Thread Details : (139781264041792, MainThread))
Log : Addition of 10 & 20 is : 30.0

16-03-2021 05:01:13 : DEBUG : logging_config_ex_1 : addition : 28 : (Process Details : (23515, MainProcess), Thread Details : (139781264041792, MainThread))
Log : Inside Addition Function
16-03-2021 05:01:13 : WARNING : logging_config_ex_1 : addition : 30 : (Process Details : (23515, MainProcess), Thread Details : (139781264041792, MainThread))
Log : Warning : Parameter A is passed as String. Future versions won't support it.
16-03-2021 05:01:13 : INFO : logging_config_ex_1 : addition : 37 : (Process Details : (23515, MainProcess), Thread Details : (139781264041792, MainThread))
Log : Addition Function Completed Successfully
16-03-2021 05:01:13 : INFO : logging_config_ex_1 : <module> : 51 : (Process Details : (23515, MainProcess), Thread Details : (139781264041792, MainThread))
Log : Addition of '20' & 20 is : 40.0

16-03-2021 05:01:13 : DEBUG : logging_config_ex_1 : addition : 28 : (Process Details : (23515, MainProcess), Thread Details : (139781264041792, MainThread))
Log : Inside Addition Function
16-03-2021 05:01:13 : ERROR : logging_config_ex_1 : addition : 40 : (Process Details : (23515, MainProcess), Thread Details : (139781264041792, MainThread))
Log : Error Type : ValueError, Error Message : could not convert string to float: 'A'
16-03-2021 05:01:13 : INFO : logging_config_ex_1 : <module> : 54 : (Process Details : (23515, MainProcess), Thread Details : (139781264041792, MainThread))
Log : Addition of A & 20 is : None

Below, we have explained important keys of dictionary configuration.

1.1 How to Define Dictionary for Configuration Schema (Important Dictionary Keys)?

The dictionary should have below a list of keys. Some of the keys are compulsory and some are optional.

  • version(compulsory) - This key accepts value 1 which is the only value available now.
  • root(optional) - This key represents the root logger of the module. When we'll create a logger at the module level, the configuration provided as the value of this key will be used to configure it. This key accepts a dictionary representing configuration and has a list of possible keys as below.
    • level(optional) - The level key accepts string specifying logging level.
    • handlers(optional) - The handlers key accepts list of handler names which are defined in handlers part of dictionary.
    • filters(optional) -The filters key accepts list of filter names which are defined in filters part of the dictionary.
  • handlers(optional) - This key represents a list of handlers that will be used by different loggers to handle log messages. It accepts another dictionary as a value where each key of that dictionary will be a handler name and value will be another dictionary defining that handler. Below is a list of keys that will be present inside of value of dictionary defining handlers.
    • class(compulsory) - This key accepts string value specifying handler class inside of logging module whose instance should be created to instantiate this handler.
    • level(optional) - This key accepts logging level name as a value specifying logging level of handler.
    • formatter(optional) - This key accepts the name of the formatter that will be used to format messages handled by this handler.
  • formatters(optional) - This key represents a list of formatters that will be used by different handlers to format messages. This key accepts another dictionary as a value where each key of that dictionary will be a formatter name and the value will be another dictionary having the configuration of the formatter. The dictionary which will be used to configure formatter can have the below keys.
    • format(optional) - This key accepts a string that will be used to format log messages by handler which uses this formatter.
    • datefmt(optional) - This key accepts a string that will be used to format the date inside of the log message.
  • filters(optional) - This key represents a list of filters that will be used to filter log messages. The filters can be attached to different handlers. The value of this key is a dictionary where each key represents the name of the filter and each value is another dictionary having the configuration of that filter. Below is a list of possible keys values for the dictionary which will be used to configure the individual filter.
    • () - This key will have a class named which will have a filter definition. The logging module will instantiate this class to create a filter.
    • All other key-value pairs will be given as parameters to the class.
  • loggers(optional) - This key represents a list of loggers that we can define with different configurations. The value of this key is another dictionary whose each key is a logger name and value is another dictionary having the configuration of that logger. The list of loggers can be defined with the same keys that we had used for defining root logger like handlers, level, filters, etc.
  • disable_existing_loggers - This key accepts boolean value True or False. The True value indicates to disable all other existing non-root loggers and False value otherwise. If this key is not provided then a default value of True is considered for it.

NOTE

Please make a NOTE that the above explanation does not cover all possible keys that can be provided to the dictionary. It's a list of commonly used possible keys.

2. Create Logger using Dictionary Config

As a part of our second example, we'll explain how we can create a logger using the default configuration.

Our code for this example is exactly the same as our code for the previous example with few minor modifications.

We have created a Logger instance using getLogger() method and we'll be using this logger to log messages. We have modified the code inside of addition() method and the main part to use this logger instance's method to log instances.

When we run the below script, the output will be exactly the same as our previous example but this time we have used logger instance to log messages instead of logging module methods.

logging_config_ex_2.py

import logging
from logging import config

log_config = {
    "version":1,
    "root":{
        "handlers" : ["console"],
        "level": "DEBUG"
    },
    "handlers":{
        "console":{
            "formatter": "std_out",
            "class": "logging.StreamHandler",
            "level": "DEBUG"
        }
    },
    "formatters":{
        "std_out": {
            "format": "%(asctime)s : %(levelname)s : %(module)s : %(funcName)s : %(lineno)d : (Process Details : (%(process)d, %(processName)s), Thread Details : (%(thread)d, %(threadName)s))\nLog : %(message)s",
            "datefmt":"%d-%m-%Y %I:%M:%S"
        }
    },
}

config.dictConfig(log_config)

################ Logger #################
logger = logging.getLogger(__name__)

def addition(a, b):
    logger.debug("Inside Addition Function")
    if isinstance(a, str) and a.isdigit():
        logger.warning("Warning : Parameter A is passed as String. Future versions won't support it.")

    if isinstance(b, str) and b.isdigit():
        logger.warning("Warning : Parameter B is passed as String. Future versions won't support it.")

    try:
        result = float(a) + float(b)
        logger.info("Addition Function Completed Successfully")
        return result
    except Exception as e:
        logger.error("Error Type : {}, Error Message : {}".format(type(e).__name__, e))
        return None


if __name__ == "__main__":
    logger.info("Current Log Level : {}\n".format(logger.getEffectiveLevel()))


    result = addition(10,20)
    logger.info("Addition of {} & {} is : {}\n".format(10,20, result))

    result = addition("20",20)
    logger.info("Addition of {} & {} is : {}\n".format("'20'",20, result))

    result = addition("A",20)
    logger.info("Addition of {} & {} is : {}".format("A",20, result))

OUTPUT

16-03-2021 05:31:32 : INFO : logging_config_ex_2 : <module> : 48 : (Process Details : (9582, MainProcess), Thread Details : (140308966008640, MainThread))
Log : Current Log Level : 10

16-03-2021 05:31:32 : DEBUG : logging_config_ex_2 : addition : 31 : (Process Details : (9582, MainProcess), Thread Details : (140308966008640, MainThread))
Log : Inside Addition Function
16-03-2021 05:31:32 : INFO : logging_config_ex_2 : addition : 40 : (Process Details : (9582, MainProcess), Thread Details : (140308966008640, MainThread))
Log : Addition Function Completed Successfully
16-03-2021 05:31:32 : INFO : logging_config_ex_2 : <module> : 52 : (Process Details : (9582, MainProcess), Thread Details : (140308966008640, MainThread))
Log : Addition of 10 & 20 is : 30.0

16-03-2021 05:31:32 : DEBUG : logging_config_ex_2 : addition : 31 : (Process Details : (9582, MainProcess), Thread Details : (140308966008640, MainThread))
Log : Inside Addition Function
16-03-2021 05:31:32 : WARNING : logging_config_ex_2 : addition : 33 : (Process Details : (9582, MainProcess), Thread Details : (140308966008640, MainThread))
Log : Warning : Parameter A is passed as String. Future versions won't support it.
16-03-2021 05:31:32 : INFO : logging_config_ex_2 : addition : 40 : (Process Details : (9582, MainProcess), Thread Details : (140308966008640, MainThread))
Log : Addition Function Completed Successfully
16-03-2021 05:31:32 : INFO : logging_config_ex_2 : <module> : 55 : (Process Details : (9582, MainProcess), Thread Details : (140308966008640, MainThread))
Log : Addition of '20' & 20 is : 40.0

16-03-2021 05:31:32 : DEBUG : logging_config_ex_2 : addition : 31 : (Process Details : (9582, MainProcess), Thread Details : (140308966008640, MainThread))
Log : Inside Addition Function
16-03-2021 05:31:32 : ERROR : logging_config_ex_2 : addition : 43 : (Process Details : (9582, MainProcess), Thread Details : (140308966008640, MainThread))
Log : Error Type : ValueError, Error Message : could not convert string to float: 'A'
16-03-2021 05:31:32 : INFO : logging_config_ex_2 : <module> : 58 : (Process Details : (9582, MainProcess), Thread Details : (140308966008640, MainThread))
Log : Addition of A & 20 is : None

3. Dictionary Config to Create Multiple Loggers

As a part of our third example, we are demonstrating how we can configure multiple loggers with different configurations using dictionary configuration.

Our dictionary for this part creates a root loggers like our previous example. We have introduced a new section named loggers where we have defined a new logger named main. We have used a different handler than used by root logger as a handler of main logger. We have also set level of main logger to INFO and propagate attribute to False. We have then created two handlers named console1 and console2. Both have the same configuration but console1 will be used by root logger and console2 will be used by main logger. We have then created two different formatters named std_out1 and std_out2 with different string formatting of log messages. The std_out1 formatter will be used by root logger and std_out2 formatter will be used by main logger.

We have then configured the logging module using this dictionary.

We have then created two loggers (logger1 and logger2). The logger1 will have configuration of root logger and logger2 will have configuration of main logger. We have modified the code inside of addition method to use logger1 to log messages. Our main part of the code uses logger2 to log messages.

When we run the below script, we can notice from the output based on different log message formats that log messages logged by addition method has the same format as for root logger and log messages logged by the main part of the code has the same format as for main logger.

logging_config_ex_3.py

import logging
from logging import config

log_config = {
    "version":1,
    "root":{
        "handlers" : ["console1"],
        "level": "DEBUG"
    },
    "loggers":{
        "main":{
            "handlers" : ["console2"],
            "level": "INFO",
            "propagate":False
        }
    },
    "handlers":{
        "console1":{
            "formatter": "std_out1",
            "class": "logging.StreamHandler",
            "level": "DEBUG"
        },
        "console2":{
            "formatter": "std_out2",
            "class": "logging.StreamHandler",
            "level": "DEBUG"
        }
    },
    "formatters":{
        "std_out1": {
            "format": "%(asctime)s : %(levelname)s : %(module)s : %(funcName)s : %(lineno)d : (Process Details : (%(process)d, %(processName)s), Thread Details : (%(thread)d, %(threadName)s))\nLog : %(message)s",
            "datefmt":"%d-%m-%Y %I:%M:%S"
        },
        "std_out2": {
            "format": "%(asctime)s : %(levelname)s : %(name)s : %(funcName)s : %(message)s",
            "datefmt":"%d-%m-%Y %I:%M:%S"
        }
    },
}

config.dictConfig(log_config)

################ Logger #################
logger1 = logging.getLogger("root")
logger2 = logging.getLogger("main")

def addition(a, b):
    logger1.debug("Inside Addition Function")
    if isinstance(a, str) and a.isdigit():
        logger1.warning("Warning : Parameter A is passed as String. Future versions won't support it.")

    if isinstance(b, str) and b.isdigit():
        logger1.warning("Warning : Parameter B is passed as String. Future versions won't support it.")

    try:
        result = float(a) + float(b)
        logger1.info("Addition Function Completed Successfully")
        return result
    except Exception as e:
        logger1.error("Error Type : {}, Error Message : {}".format(type(e).__name__, e))
        return None


if __name__ == "__main__":
    logger2.info("Current Log Level : {}\n".format(logger2.getEffectiveLevel()))


    result = addition(10,20)
    logger2.info("Addition of {} & {} is : {}\n".format(10,20, result))

    result = addition("20",20)
    logger2.info("Addition of {} & {} is : {}\n".format("'20'",20, result))

    result = addition("A",20)
    logger2.info("Addition of {} & {} is : {}".format("A",20, result))

OUTPUT

16-03-2021 05:32:36 : INFO : main : <module> : Current Log Level : 20

16-03-2021 05:32:36 : DEBUG : logging_config_ex_3 : addition : 48 : (Process Details : (12751, MainProcess), Thread Details : (140392382891840, MainThread))
Log : Inside Addition Function
16-03-2021 05:32:36 : INFO : logging_config_ex_3 : addition : 57 : (Process Details : (12751, MainProcess), Thread Details : (140392382891840, MainThread))
Log : Addition Function Completed Successfully
16-03-2021 05:32:36 : INFO : main : <module> : Addition of 10 & 20 is : 30.0

16-03-2021 05:32:36 : DEBUG : logging_config_ex_3 : addition : 48 : (Process Details : (12751, MainProcess), Thread Details : (140392382891840, MainThread))
Log : Inside Addition Function
16-03-2021 05:32:36 : WARNING : logging_config_ex_3 : addition : 50 : (Process Details : (12751, MainProcess), Thread Details : (140392382891840, MainThread))
Log : Warning : Parameter A is passed as String. Future versions won't support it.
16-03-2021 05:32:36 : INFO : logging_config_ex_3 : addition : 57 : (Process Details : (12751, MainProcess), Thread Details : (140392382891840, MainThread))
Log : Addition Function Completed Successfully
16-03-2021 05:32:36 : INFO : main : <module> : Addition of '20' & 20 is : 40.0

16-03-2021 05:32:36 : DEBUG : logging_config_ex_3 : addition : 48 : (Process Details : (12751, MainProcess), Thread Details : (140392382891840, MainThread))
Log : Inside Addition Function
16-03-2021 05:32:36 : ERROR : logging_config_ex_3 : addition : 60 : (Process Details : (12751, MainProcess), Thread Details : (140392382891840, MainThread))
Log : Error Type : ValueError, Error Message : could not convert string to float: 'A'
16-03-2021 05:32:36 : INFO : main : <module> : Addition of A & 20 is : None

4. Dictionary Config to Direct Logs to File and Console

As a part of our fourth example, we are demonstrating how we can direct log messages to the console and to a file using dictionary configuration.

Our configuration dictionary for this part creates a root logger with two handlers (console and file). The handler section has defined two handlers. The console handler is the same as our previous examples. The file handler is defined with logging.FileHandler class. It has the logging level set to INFO and log file name set to all_message.log to which log messages will be directed. We have modified the format of our std_out formatter to include few details, unlike our previous examples. This formatter will be used by our both handlers to log messages.

Our code for this example is exactly same as our code for the second example. It created a single logger and used it to log all messages.

When we run the below script, we can notice from the output that all log messages with level DEBUG and above are directed to standard output. We have also displayed the contents of all_messages.log file where all log messages with level INFO and above are directed.

logging_config_ex_4.py

import logging
from logging import config

log_config = {
    "version":1,
    "root":{
        "handlers" : ["console", "file"],
        "level": "DEBUG"
    },
    "handlers":{
        "console":{
            "formatter": "std_out",
            "class": "logging.StreamHandler",
            "level": "DEBUG"
        },
        "file":{
            "formatter":"std_out",
            "class":"logging.FileHandler",
            "level":"INFO",
            "filename":"all_messages.log"
        }
    },
    "formatters":{
        "std_out": {
            "format": "%(levelname)s : %(module)s : %(funcName)s : %(message)s",
        }
    },
}

config.dictConfig(log_config)

################ Logger #################
logger = logging.getLogger(__name__)

def addition(a, b):
    logger.debug("Inside Addition Function")
    if isinstance(a, str) and a.isdigit():
        logger.warning("Warning : Parameter A is passed as String. Future versions won't support it.")

    if isinstance(b, str) and b.isdigit():
        logger.warning("Warning : Parameter B is passed as String. Future versions won't support it.")

    try:
        result = float(a) + float(b)
        logger.info("Addition Function Completed Successfully")
        return result
    except Exception as e:
        logger.error("Error Type : {}, Error Message : {}".format(type(e).__name__, e))
        return None


if __name__ == "__main__":
    logger.info("Current Log Level : {}\n".format(logger.getEffectiveLevel()))


    result = addition(10,20)
    logger.info("Addition of {} & {} is : {}\n".format(10,20, result))

    result = addition("20",20)
    logger.info("Addition of {} & {} is : {}\n".format("'20'",20, result))

    result = addition("A",20)
    logger.info("Addition of {} & {} is : {}".format("A",20, result))

OUTPUT

INFO : logging_config_ex_4 : <module> : Current Log Level : 10

DEBUG : logging_config_ex_4 : addition : Inside Addition Function
INFO : logging_config_ex_4 : addition : Addition Function Completed Successfully
INFO : logging_config_ex_4 : <module> : Addition of 10 & 20 is : 30.0

DEBUG : logging_config_ex_4 : addition : Inside Addition Function
WARNING : logging_config_ex_4 : addition : Warning : Parameter A is passed as String. Future versions won't support it.
INFO : logging_config_ex_4 : addition : Addition Function Completed Successfully
INFO : logging_config_ex_4 : <module> : Addition of '20' & 20 is : 40.0

DEBUG : logging_config_ex_4 : addition : Inside Addition Function
ERROR : logging_config_ex_4 : addition : Error Type : ValueError, Error Message : could not convert string to float: 'A'
INFO : logging_config_ex_4 : <module> : Addition of A & 20 is : None
!cat logging_config_examples/all_messages.log
INFO : logging_config_ex_4 : <module> : Current Log Level : 10

INFO : logging_config_ex_4 : addition : Addition Function Completed Successfully
INFO : logging_config_ex_4 : <module> : Addition of 10 & 20 is : 30.0

WARNING : logging_config_ex_4 : addition : Warning : Parameter A is passed as String. Future versions won't support it.
INFO : logging_config_ex_4 : addition : Addition Function Completed Successfully
INFO : logging_config_ex_4 : <module> : Addition of '20' & 20 is : 40.0

ERROR : logging_config_ex_4 : addition : Error Type : ValueError, Error Message : could not convert string to float: 'A'
INFO : logging_config_ex_4 : <module> : Addition of A & 20 is : None

5. Dictionary Config for Hierarchy Of Loggers

As a part of our fifth example, we are explaining how we can create a hierarchy of loggers.

We have a configuration dictionary for this example which is exactly the same as our previous example with only a change in the message format.

We have moved our code for addition method to add() method of the class Addition. Our main part of the code creates an instance of Addition and performs addition using it's add() method.

We have created a logger with the name arithmetic_ops which will be used to log messages inside the main part of our code. We have then created another logger named arithmetic_ops.Addition will be used to log messages inside of add() method. The logger created inside of Addition class will be sub logger of our main logger (arithmetic_ops).

When we run the below code, we can notice from the output that log messages logged by the main logger have the name arithmetic_ops and log messages logged by sub-logger have the name arithmetic_ops.Addition.

logging_config_ex_5.py

import logging
from logging import config

log_config = {
    "version":1,
    "root":{
        "handlers" : ["console"],
        "level": "DEBUG"
    },
    "handlers":{
        "console":{
            "formatter": "std_out",
            "class": "logging.StreamHandler",
            "level": "DEBUG"
        },
    },
    "formatters":{
        "std_out": {
            "format": "%(levelname)s : %(name)s : %(funcName)s : %(message)s",
        }
    },
}

config.dictConfig(log_config)

################ Module Logger #################
logger = logging.getLogger("arithmetic_ops")

class Addition:
    def __init__(self):
        ################ Class Logger #################
        self.logger = logging.getLogger("arithmetic_ops.Addition")

    def add(self, a,b):
        self.logger.debug("Inside Addition Function")
        if isinstance(a, str) and a.isdigit():
            self.logger.warning("Warning : Parameter A is passed as String. Future versions won't support it.")

        if isinstance(b, str) and b.isdigit():
            self.logger.warning("Warning : Parameter B is passed as String. Future versions won't support it.")

        try:
            result = float(a) + float(b)
            self.logger.info("Addition Function Completed Successfully")
            return result
        except Exception as e:
            self.logger.error("Error Type : {}, Error Message : {}".format(type(e).__name__, e))
            return None


if __name__ == "__main__":
    logger.info("Current Log Level : {}\n".format(logger.getEffectiveLevel()))

    addition = Addition()
    result = addition.add(10,20)
    logger.info("Addition of {} & {} is : {}\n".format(10,20, result))

    result = addition.add("20",20)
    logger.info("Addition of {} & {} is : {}\n".format("'20'",20, result))

    result = addition.add("A",20)
    logger.info("Addition of {} & {} is : {}".format("A",20, result))

OUTPUT

INFO : arithmetic_ops : <module> : Current Log Level : 10

DEBUG : arithmetic_ops.Addition : add : Inside Addition Function
INFO : arithmetic_ops.Addition : add : Addition Function Completed Successfully
INFO : arithmetic_ops : <module> : Addition of 10 & 20 is : 30.0

DEBUG : arithmetic_ops.Addition : add : Inside Addition Function
WARNING : arithmetic_ops.Addition : add : Warning : Parameter A is passed as String. Future versions won't support it.
INFO : arithmetic_ops.Addition : add : Addition Function Completed Successfully
INFO : arithmetic_ops : <module> : Addition of '20' & 20 is : 40.0

DEBUG : arithmetic_ops.Addition : add : Inside Addition Function
ERROR : arithmetic_ops.Addition : add : Error Type : ValueError, Error Message : could not convert string to float: 'A'
INFO : arithmetic_ops : <module> : Addition of A & 20 is : None

6. Dictionary Config to Include Filters

As a part of our sixth example, we are demonstrating how we can include filters inside of dictionary configuration.

We have created a class named InfoAnWarningsOnly by extending logging.Filter class which implements filter() method. The filter() method takes as input LogRecord instance. The method checks if the log level is between 10-30 then it returns True else it returns False. It also modifies the log message of logs with level 30 (WARNING) to include the string Error : to indicate that they are errors going forward.

Our logging dictionary creates a root logger with console handler, info_&_warnings filter and log level DEBUG. The definition of console handler and formatters is the same as our previous examples. We have included info_&_warnings filter inside of console handler as well. We have included a new section named filters in the configuration dictionary this time. We have set () parameter to InfoAnWarningsOnly filter class which we had defined earlier.

Other than dictionary configuration our rest of the code is the same as our code from examples 2 and 4.

When we run the below script, we can notice from the output that all messages except INFO and WARNING are filtered. Apart from this, the WARNING messages are modified to include Error : string.

logging_config_ex_6.py

import logging
from logging import config

class InfoAnWarningsOnly(logging.Filter):
    def filter(self, record):
        if record.levelno >10 and record.levelno < 31:
            if record.levelno == 30: ## Updating Warning messages to Error.
                record.levelno = 40
                record.msg = "Error : " + record.msg
            return True
        else:
            return False

log_config = {
    "version":1,
    "root":{
        "handlers" : ["console"],
        "filters": ["info_&_warnings"],
        "level": "DEBUG"
    },
    "handlers":{
        "console":{
            "formatter": "std_out",
            "filters": ["info_&_warnings"],
            "class": "logging.StreamHandler",
            "level": "DEBUG"
        }
    },
    "formatters":{
        "std_out": {
            "format": "%(levelname)s : %(name)s : %(funcName)s : %(message)s",
            "datefmt":"%d-%m-%Y %I:%M:%S"
        }
    },
    "filters":{
        "info_&_warnings":{
            "()": InfoAnWarningsOnly
        }

    }
}

config.dictConfig(log_config)

################ Logger #################
logger = logging.getLogger("root")

def addition(a, b):
    logger.debug("Inside Addition Function")
    if isinstance(a, str) and a.isdigit():
        logger.warning("Warning : Parameter A is passed as String. Future versions won't support it.")

    if isinstance(b, str) and b.isdigit():
        logger.warning("Warning : Parameter B is passed as String. Future versions won't support it.")

    try:
        result = float(a) + float(b)
        logger.info("Addition Function Completed Successfully")
        return result
    except Exception as e:
        logger.error("Error Type : {}, Error Message : {}".format(type(e).__name__, e))
        return None


if __name__ == "__main__":
    logger.info("Current Log Level : {}\n".format(logger.getEffectiveLevel()))


    result = addition(10,20)
    logger.info("Addition of {} & {} is : {}\n".format(10,20, result))

    result = addition("20",20)
    logger.info("Addition of {} & {} is : {}\n".format("'20'",20, result))

    result = addition("A",20)
    logger.info("Addition of {} & {} is : {}".format("A",20, result))

OUTPUT

INFO : root : <module> : Current Log Level : 10

INFO : root : addition : Addition Function Completed Successfully
INFO : root : <module> : Addition of 10 & 20 is : 30.0

WARNING : root : addition : Error : Warning : Parameter A is passed as String. Future versions won't support it.
INFO : root : addition : Addition Function Completed Successfully
INFO : root : <module> : Addition of '20' & 20 is : 40.0

INFO : root : <module> : Addition of A & 20 is : None

7. Dictionary Config for Multiple Filters

As a part of our seventh example, we are demonstrating how we can use more than one filter with dictionary configuration.

We have defined three classes for implementing filters.

  • DebugAndInfoOnly - This class filters all log messages except DEBUG and INFO level messages.
  • WarningsOnly - This class filters all log messages except WARNING level messages.
  • ErrorsOnly - This class filters all log messages except ERROR level messages.

Our dictionary configuration starts by defining root logger with three handlers (console, warnings_handler and errors_handler). The console handler directs log messages to standard output and uses a filter named debug_&_info. The warnings_handler handler directs log messages to a file named warnings_only.log and used warnings_only filter. The warnings_handler handler directs log messages to a file named errors_only.log and used errors_only filter. The std_out is the formatter that will be used by all handlers. We have then defined all three filters inside of filters section.

Our rest of the code is exactly the same as our code for the previous example.

When we run the below code, we can notice that only INFO and DEBUG messages are directed to standard output. We have also displayed contents of warnings_only.log file which holds all warning messages and contents of errors)only.log file which holds all error messages.

logging_config_ex_7.py

import logging
from logging import config

class DebugAndInfoOnly(logging.Filter): ### Filters all messages except Info and Debug
    def filter(self, record):
        if record.levelno < 21:
            return True
        else:
            return False

class WarningsOnly(logging.Filter): ### Filters all messages except Warning
    def filter(self, record):
        if record.levelno > 20 and record.levelno < 40:
            return True
        else:
            return False

class ErrorsOnly(logging.Filter): ### Filters all messages except Error
    def filter(self, record):
        if record.levelno > 30 and record.levelno < 50:
            return True
        else:
            return False

log_config = {
    "version":1,
    "root":{
        "handlers" : ["console", "warnings_handler", "errors_handler"],
        "level": "DEBUG"
    },
    "handlers":{
        "console":{
            "formatter": "std_out",
            "filters": ["debug_&_info"],
            "class": "logging.StreamHandler",
            "level": "DEBUG"
        },
        "warnings_handler":{
            "formatter":"std_out",
            "class":"logging.FileHandler",
            "filters": ["warnings"],
            "level":"DEBUG",
            "filename":"warnings_only.log"
        },
        "errors_handler":{
            "formatter":"std_out",
            "class":"logging.FileHandler",
            "filters": ["errors"],
            "level":"DEBUG",
            "filename":"errors_only.log"
        },
    },
    "formatters":{
        "std_out": {
            "format": "%(levelname)s : %(name)s : %(funcName)s : %(message)s",
            "datefmt":"%d-%m-%Y %I:%M:%S"
        }

    },
    "filters":{
        "debug_&_info":{
            "()": DebugAndInfoOnly
        },
        "warnings":{
            "()": WarningsOnly
        },
        "errors":{
            "()": ErrorsOnly
        }

    }
}

config.dictConfig(log_config)

################ Logger #################
logger = logging.getLogger("root")

def addition(a, b):
    logger.debug("Inside Addition Function")
    if isinstance(a, str) and a.isdigit():
        logger.warning("Warning : Parameter A is passed as String. Future versions won't support it.")

    if isinstance(b, str) and b.isdigit():
        logger.warning("Warning : Parameter B is passed as String. Future versions won't support it.")

    try:
        result = float(a) + float(b)
        logger.info("Addition Function Completed Successfully")
        return result
    except Exception as e:
        logger.error("Error Type : {}, Error Message : {}".format(type(e).__name__, e))
        return None


if __name__ == "__main__":
    logger.info("Current Log Level : {}\n".format(logger.getEffectiveLevel()))


    result = addition(10,20)
    logger.info("Addition of {} & {} is : {}\n".format(10,20, result))

    result = addition("20",20)
    logger.info("Addition of {} & {} is : {}\n".format("'20'",20, result))

    result = addition("A",20)
    logger.info("Addition of {} & {} is : {}".format("A",20, result))

OUTPUT

INFO : root : <module> : Current Log Level : 10

DEBUG : root : addition : Inside Addition Function
INFO : root : addition : Addition Function Completed Successfully
INFO : root : <module> : Addition of 10 & 20 is : 30.0

DEBUG : root : addition : Inside Addition Function
INFO : root : addition : Addition Function Completed Successfully
INFO : root : <module> : Addition of '20' & 20 is : 40.0

DEBUG : root : addition : Inside Addition Function
INFO : root : <module> : Addition of A & 20 is : None
!cat logging_config_examples/warnings_only.log
WARNING : root : addition : Warning : Parameter A is passed as String. Future versions won't support it.
!cat logging_config_examples/errors_only.log
ERROR : root : addition : Error Type : ValueError, Error Message : could not convert string to float: 'A'

2. File Config Examples

1. File Config to Log Messages with Level DEBUG & Above

All our examples from example eighth to the end will be used to explain how we can use configuration files to configure the logging module in Python. Our examples from this till the end will be almost the same as our previous dictionary configuration examples with the only change that we'll use the file to configure the logging module instead of a dictionary.


  • fileConfig(filename,defaults=None,disable_existing_loggers=True) - This method takes as input configuration file name and configures logging module based on it.
    • It has a parameter named defaults which accepts a dictionary with a list of key-value pairs. This dictionary will be used when parsing a configuration file if some key is expected in some section but is not present there then the value of that key will be taken from this default dictionary.

NOTE

Please make a NOTE that Python internally uses configparser module to parse the configuration file. If you are interested in learning about configparser module then please feel free to check our tutorial on the same.


As a part of our eighth example, we are demonstrating how we can configure the logging module with a configuration file. The configuration file follows a particular format which we need to follow to configure the module (covered below after example output).

First, we have created a configuration file named file_config1.conf which will be used to configure logging.

We have defined a single logger named root inside of the configuration file. We have then defined one handler named console and one formatter named std_out. We have then included a definition of root logger to have log level as DEBUG and handler as console. We have then included a definition of console handler which uses logging.StreamHandler to log messages to standard output with a log level of DEBUG and formatter Std_out. At last, we have included a formatter definition for std_out formatter. This file has exactly the same configuration as that we created in our first example using a dictionary.

Our script starts by configuring the logging module with file_config1.conf file. All the remaining code is exactly the same as our first example.

When we run the script, we can notice from the output that it's exactly the same as our first example.

file_config1.conf

[loggers]
keys=root

[handlers]
keys=console

[formatters]
keys=std_out

[logger_root]
handlers = console
level = DEBUG

[handler_console]
class = logging.StreamHandler
level = DEBUG
formatter = std_out

[formatter_std_out]
format = %(asctime)s : %(levelname)s : %(module)s : %(funcName)s : %(lineno)d : (Process Details : (%(process)d, %(processName)s), Thread Details : (%(thread)d, %(threadName)s))\nLog : %(message)s
datefmt = %d-%m-%Y %I:%M:%S

logging_config_ex_8.py

import logging
from logging import config



config.fileConfig("file_config1.conf")

def addition(a, b):
    logging.debug("Inside Addition Function")
    if isinstance(a, str) and a.isdigit():
        logging.warning("Warning : Parameter A is passed as String. Future versions won't support it.")

    if isinstance(b, str) and b.isdigit():
        logging.warning("Warning : Parameter B is passed as String. Future versions won't support it.")

    try:
        result = float(a) + float(b)
        logging.info("Addition Function Completed Successfully")
        return result
    except Exception as e:
        logging.error("Error Type : {}, Error Message : {}".format(type(e).__name__, e))
        return None


if __name__ == "__main__":
    #logging.info("Current Log Level : {}\n".format(logging.getLevel()))

    result = addition(10,20)
    logging.info("Addition of {} & {} is : {}\n".format(10,20, result))

    result = addition("20",20)
    logging.info("Addition of {} & {} is : {}\n".format("'20'",20, result))

    result = addition("A",20)
    logging.info("Addition of {} & {} is : {}".format("A",20, result))

OUTPUT

18-03-2021 05:31:36 : DEBUG : logging_config_ex_8 : addition : 9 : (Process Details : (9368, MainProcess), Thread Details : (140332160935744, MainThread))\nLog : Inside Addition Function
18-03-2021 05:31:36 : INFO : logging_config_ex_8 : addition : 18 : (Process Details : (9368, MainProcess), Thread Details : (140332160935744, MainThread))\nLog : Addition Function Completed Successfully
18-03-2021 05:31:36 : INFO : logging_config_ex_8 : <module> : 29 : (Process Details : (9368, MainProcess), Thread Details : (140332160935744, MainThread))\nLog : Addition of 10 & 20 is : 30.0

18-03-2021 05:31:36 : DEBUG : logging_config_ex_8 : addition : 9 : (Process Details : (9368, MainProcess), Thread Details : (140332160935744, MainThread))\nLog : Inside Addition Function
18-03-2021 05:31:36 : WARNING : logging_config_ex_8 : addition : 11 : (Process Details : (9368, MainProcess), Thread Details : (140332160935744, MainThread))\nLog : Warning : Parameter A is passed as String. Future versions won't support it.
18-03-2021 05:31:36 : INFO : logging_config_ex_8 : addition : 18 : (Process Details : (9368, MainProcess), Thread Details : (140332160935744, MainThread))\nLog : Addition Function Completed Successfully
18-03-2021 05:31:36 : INFO : logging_config_ex_8 : <module> : 32 : (Process Details : (9368, MainProcess), Thread Details : (140332160935744, MainThread))\nLog : Addition of '20' & 20 is : 40.0

18-03-2021 05:31:36 : DEBUG : logging_config_ex_8 : addition : 9 : (Process Details : (9368, MainProcess), Thread Details : (140332160935744, MainThread))\nLog : Inside Addition Function
18-03-2021 05:31:36 : ERROR : logging_config_ex_8 : addition : 21 : (Process Details : (9368, MainProcess), Thread Details : (140332160935744, MainThread))\nLog : Error Type : ValueError, Error Message : could not convert string to float: 'A'
18-03-2021 05:31:36 : INFO : logging_config_ex_8 : <module> : 35 : (Process Details : (9368, MainProcess), Thread Details : (140332160935744, MainThread))\nLog : Addition of A & 20 is : None

1.1 Configuration File Format Explanation


The configuration file should have the below-mentioned three sections which will be used to define the names of the list of loggers, handlers, and formatters.

[loggers]
keys=root,logger1,logger2,...

[handlers]
keys=handler1,handler2,handler3,..

[formatters]
keys=formatter1,formatter2,formatter3,...

The three sections will define a list of loggers, the list of handlers, and a list of formatters that will be present in this configuration file.

After these three sections, there will be a list of sections defining individual logger, individual handler, and individual formatter as explained below.

[logger_root]
level=DEBUG
handler=handler1

[logger_logger1]
level=DEBUG
handler=console

[logger_logger2]
level=DEBUG
handler=handler2
.
.
.

[handler_handler1]
class = logging.StreamHandler
level = DEBUG
formatter = formatter1

[handler_handler2]
class = logging.StreamHandler
level = DEBUG
formatter = formatter2

[handler_handler3]
class = logging.StreamHandler
level = DEBUG
formatter = formatter3
.
.
.

[formatter_formatter1]
format = %(levelname)s : %(module)s : %(funcName)s : %(lineno)d : %(message)s
datefmt = %d-%m-%Y %I:%M:%S

[formatter_formatter2]
format = %(levelname)s : %(module)s : %(funcName)s : %(lineno)d : %(message)s
datefmt = %d-%m-%Y %I:%M:%S
.
.
.

2. File Config to Create Logger

As a part of our ninth example, we are explaining how we can create loggers to log messages from the logging module which is configured from the configuration file.

Our code for this example uses the same configuration file file_config1.conf which we have used in our previous example. The rest of our code is exactly the same as our code for the second example.

When we run the below script, we can notice that the output is exactly the same as our output from the second example.

logging_config_ex_9.py

import logging
from logging import config



config.fileConfig("file_config1.conf")

logger = logging.getLogger(__name__)

def addition(a, b):
    logger.debug("Inside Addition Function")
    if isinstance(a, str) and a.isdigit():
        logger.warning("Warning : Parameter A is passed as String. Future versions won't support it.")

    if isinstance(b, str) and b.isdigit():
        logger.warning("Warning : Parameter B is passed as String. Future versions won't support it.")

    try:
        result = float(a) + float(b)
        logger.info("Addition Function Completed Successfully")
        return result
    except Exception as e:
        logger.error("Error Type : {}, Error Message : {}".format(type(e).__name__, e))
        return None


if __name__ == "__main__":
    logger.info("Current Log Level : {}\n".format(logger.getEffectiveLevel()))

    result = addition(10,20)
    logger.info("Addition of {} & {} is : {}\n".format(10,20, result))

    result = addition("20",20)
    logger.info("Addition of {} & {} is : {}\n".format("'20'",20, result))

    result = addition("A",20)
    logger.info("Addition of {} & {} is : {}".format("A",20, result))

OUTPUT

18-03-2021 05:32:37 : INFO : logging_config_ex_9 : <module> : 28 : (Process Details : (12145, MainProcess), Thread Details : (140332311365440, MainThread))\nLog : Current Log Level : 10

18-03-2021 05:32:37 : DEBUG : logging_config_ex_9 : addition : 11 : (Process Details : (12145, MainProcess), Thread Details : (140332311365440, MainThread))\nLog : Inside Addition Function
18-03-2021 05:32:37 : INFO : logging_config_ex_9 : addition : 20 : (Process Details : (12145, MainProcess), Thread Details : (140332311365440, MainThread))\nLog : Addition Function Completed Successfully
18-03-2021 05:32:37 : INFO : logging_config_ex_9 : <module> : 31 : (Process Details : (12145, MainProcess), Thread Details : (140332311365440, MainThread))\nLog : Addition of 10 & 20 is : 30.0

18-03-2021 05:32:37 : DEBUG : logging_config_ex_9 : addition : 11 : (Process Details : (12145, MainProcess), Thread Details : (140332311365440, MainThread))\nLog : Inside Addition Function
18-03-2021 05:32:37 : WARNING : logging_config_ex_9 : addition : 13 : (Process Details : (12145, MainProcess), Thread Details : (140332311365440, MainThread))\nLog : Warning : Parameter A is passed as String. Future versions won't support it.
18-03-2021 05:32:37 : INFO : logging_config_ex_9 : addition : 20 : (Process Details : (12145, MainProcess), Thread Details : (140332311365440, MainThread))\nLog : Addition Function Completed Successfully
18-03-2021 05:32:37 : INFO : logging_config_ex_9 : <module> : 34 : (Process Details : (12145, MainProcess), Thread Details : (140332311365440, MainThread))\nLog : Addition of '20' & 20 is : 40.0

18-03-2021 05:32:37 : DEBUG : logging_config_ex_9 : addition : 11 : (Process Details : (12145, MainProcess), Thread Details : (140332311365440, MainThread))\nLog : Inside Addition Function
18-03-2021 05:32:37 : ERROR : logging_config_ex_9 : addition : 23 : (Process Details : (12145, MainProcess), Thread Details : (140332311365440, MainThread))\nLog : Error Type : ValueError, Error Message : could not convert string to float: 'A'
18-03-2021 05:32:37 : INFO : logging_config_ex_9 : <module> : 37 : (Process Details : (12145, MainProcess), Thread Details : (140332311365440, MainThread))\nLog : Addition of A & 20 is : None

3. File Config to Create Different Handlers (Direct Logs to File)

As a part of our tenth example, we have explained how we can configure two different handlers with different settings using a configuration file.

We have created a new configuration file named file_config2.conf. We have included two handlers this time. The console handler will be used to direct messages to standard output. The file handler will be used to direct log messages to a file named all_message_conf.log. We have defined file handler in section handler_file with a class named logging.FileHandler to direct log messages to file all_messages_conf.log. The rest of the configuration is almost the same as our previous examples.

Our script for this example is exactly the same as our script for the previous example with the only difference that we have used file_config2.conf file to configure logging module.

When we run the below script, we can notice that output includes all log messages at level DEBUG and above. We have also displayed the contents of all_messages_conf.log file which has included all messages at level INFO and above as per configuration.

file_config2.conf

[loggers]
keys=root

[handlers]
keys=console, file

[formatters]
keys=std_out

[logger_root]
handlers = console, file
level = DEBUG

[handler_console]
class = logging.StreamHandler
level = DEBUG
formatter = std_out


[handler_file]
class = logging.FileHandler
kwargs = {"filename": "all_messages_conf.log"}
level = INFO
formatter = std_out

[formatter_std_out]
format = %(levelname)s : %(name)s : %(module)s : %(funcName)s : %(message)s

logging_config_ex_10.py

import logging
from logging import config



config.fileConfig("file_config2.conf")

logger = logging.getLogger("root")

def addition(a, b):
    logger.debug("Inside Addition Function")
    if isinstance(a, str) and a.isdigit():
        logger.warning("Warning : Parameter A is passed as String. Future versions won't support it.")

    if isinstance(b, str) and b.isdigit():
        logger.warning("Warning : Parameter B is passed as String. Future versions won't support it.")

    try:
        result = float(a) + float(b)
        logger.info("Addition Function Completed Successfully")
        return result
    except Exception as e:
        logger.error("Error Type : {}, Error Message : {}".format(type(e).__name__, e))
        return None


if __name__ == "__main__":
    logger.info("Current Log Level : {}\n".format(logger.getEffectiveLevel()))

    result = addition(10,20)
    logger.info("Addition of {} & {} is : {}\n".format(10,20, result))

    result = addition("20",20)
    logger.info("Addition of {} & {} is : {}\n".format("'20'",20, result))

    result = addition("A",20)
    logger.info("Addition of {} & {} is : {}".format("A",20, result))

OUTPUT

INFO : root : logging_config_ex_10 : <module> : Current Log Level : 10

DEBUG : root : logging_config_ex_10 : addition : Inside Addition Function
INFO : root : logging_config_ex_10 : addition : Addition Function Completed Successfully
INFO : root : logging_config_ex_10 : <module> : Addition of 10 & 20 is : 30.0

DEBUG : root : logging_config_ex_10 : addition : Inside Addition Function
WARNING : root : logging_config_ex_10 : addition : Warning : Parameter A is passed as String. Future versions won't support it.
INFO : root : logging_config_ex_10 : addition : Addition Function Completed Successfully
INFO : root : logging_config_ex_10 : <module> : Addition of '20' & 20 is : 40.0

DEBUG : root : logging_config_ex_10 : addition : Inside Addition Function
ERROR : root : logging_config_ex_10 : addition : Error Type : ValueError, Error Message : could not convert string to float: 'A'
INFO : root : logging_config_ex_10 : <module> : Addition of A & 20 is : None
!cat logging_config_examples/all_messages_conf.log
INFO : root : logging_config_ex_10 : <module> : Current Log Level : 10

INFO : root : logging_config_ex_10 : addition : Addition Function Completed Successfully
INFO : root : logging_config_ex_10 : <module> : Addition of 10 & 20 is : 30.0

WARNING : root : logging_config_ex_10 : addition : Warning : Parameter A is passed as String. Future versions won't support it.
INFO : root : logging_config_ex_10 : addition : Addition Function Completed Successfully
INFO : root : logging_config_ex_10 : <module> : Addition of '20' & 20 is : 40.0

ERROR : root : logging_config_ex_10 : addition : Error Type : ValueError, Error Message : could not convert string to float: 'A'
INFO : root : logging_config_ex_10 : <module> : Addition of A & 20 is : None

4. File Config to Create Hierarchy of Loggers

As a part of our eleventh example, we are explaining how we can create a hierarchy of loggers for logging messages.

Our code for this example is exactly the same as our code for example 5 which has the same code but is configured using a dictionary. We have used our configuration file named file_config2.conf to configure logging in this example.

When we run the below script, we can notice based on log messages which log events are logged by which logger based on logger name present in the log message.

logging_config_ex_11.py

import logging
from logging import config

config.fileConfig("file_config2.conf")

################ Module Logger #################
logger = logging.getLogger("arithmetic_ops")

class Addition:
    def __init__(self):
        ################ Class Logger #################
        self.logger = logging.getLogger("arithmetic_ops.Addition")

    def add(self, a,b):
        self.logger.debug("Inside Addition Function")
        if isinstance(a, str) and a.isdigit():
            self.logger.warning("Warning : Parameter A is passed as String. Future versions won't support it.")

        if isinstance(b, str) and b.isdigit():
            self.logger.warning("Warning : Parameter B is passed as String. Future versions won't support it.")

        try:
            result = float(a) + float(b)
            self.logger.info("Addition Function Completed Successfully")
            return result
        except Exception as e:
            self.logger.error("Error Type : {}, Error Message : {}".format(type(e).__name__, e))
            return None


if __name__ == "__main__":
    logger.info("Current Log Level : {}\n".format(logger.getEffectiveLevel()))

    addition = Addition()
    result = addition.add(10,20)
    logger.info("Addition of {} & {} is : {}\n".format(10,20, result))

    result = addition.add("20",20)
    logger.info("Addition of {} & {} is : {}\n".format("'20'",20, result))

    result = addition.add("A",20)
    logger.info("Addition of {} & {} is : {}".format("A",20, result))

OUTPUT

INFO : arithmetic_ops : logging_config_ex_11 : <module> : Current Log Level : 10

DEBUG : arithmetic_ops.Addition : logging_config_ex_11 : add : Inside Addition Function
INFO : arithmetic_ops.Addition : logging_config_ex_11 : add : Addition Function Completed Successfully
INFO : arithmetic_ops : logging_config_ex_11 : <module> : Addition of 10 & 20 is : 30.0

DEBUG : arithmetic_ops.Addition : logging_config_ex_11 : add : Inside Addition Function
WARNING : arithmetic_ops.Addition : logging_config_ex_11 : add : Warning : Parameter A is passed as String. Future versions won't support it.
INFO : arithmetic_ops.Addition : logging_config_ex_11 : add : Addition Function Completed Successfully
INFO : arithmetic_ops : logging_config_ex_11 : <module> : Addition of '20' & 20 is : 40.0

DEBUG : arithmetic_ops.Addition : logging_config_ex_11 : add : Inside Addition Function
ERROR : arithmetic_ops.Addition : logging_config_ex_11 : add : Error Type : ValueError, Error Message : could not convert string to float: 'A'
INFO : arithmetic_ops : logging_config_ex_11 : <module> : Addition of A & 20 is : None

5. File Config to Create Multiple Loggers

As a part of our twelfth example, we are demonstrating how we can use more than one logger instance to log messages inside of the script based on configuration details from the file.

We have created a new configuration file named file_config3.conf for defining a configuration. We have created two loggers root and main. The root logger has the same configuration which we have been using for many of our examples. The main logger directs log messages at level INFO and above to standard output. We have defined two different handlers (console1 and console2) to be used by two different loggers. We have also defined two different formatters (std_out1 and std_out2) to be used by two different handlers.

Our code for this example is same as our code for previous examples with minor changes. We have created two loggers this time instead of one. We are using logger1 inside of addition method whereas we are using logger2 for the main part of our code.

When we run our script for this example, we can notice from the output which logger has logged which log message.

file_config3.conf

[loggers]
keys=root, main

[handlers]
keys=console1, console2

[formatters]
keys=std_out1, std_out2

[logger_root]
handlers = console1
level = DEBUG

[logger_main]
handlers = console2
level = INFO
qualname = main
propagate = 0

[handler_console1]
class = logging.StreamHandler
level = DEBUG
formatter = std_out1

[handler_console2]
class = logging.StreamHandler
level = INFO
formatter = std_out2

[formatter_std_out1]
format = %(levelname)s : %(name)s :%(module)s : %(funcName)s : %(message)s

[formatter_std_out2]
format = %(levelname)s : %(name)s : %(module)s : %(funcName)s : %(message)s

logging_config_ex_12.py

import logging
from logging import config



config.fileConfig("file_config3.conf")

################ Logger #################
logger1 = logging.getLogger("root")
logger2 = logging.getLogger("main")

def addition(a, b):
    logger1.debug("Inside Addition Function")
    if isinstance(a, str) and a.isdigit():
        logger1.warning("Warning : Parameter A is passed as String. Future versions won't support it.")

    if isinstance(b, str) and b.isdigit():
        logger1.warning("Warning : Parameter B is passed as String. Future versions won't support it.")

    try:
        result = float(a) + float(b)
        logger1.info("Addition Function Completed Successfully")
        return result
    except Exception as e:
        logger1.error("Error Type : {}, Error Message : {}".format(type(e).__name__, e))
        return None


if __name__ == "__main__":
    logger2.info("Current Log Level : {}\n".format(logger2.getEffectiveLevel()))


    result = addition(10,20)
    logger2.info("Addition of {} & {} is : {}\n".format(10,20, result))

    result = addition("20",20)
    logger2.info("Addition of {} & {} is : {}\n".format("'20'",20, result))

    result = addition("A",20)
    logger2.info("Addition of {} & {} is : {}".format("A",20, result))

OUTPUT

INFO : main : logging_config_ex_12 : <module> : Current Log Level : 20

DEBUG : root :logging_config_ex_12 : addition : Inside Addition Function
INFO : root :logging_config_ex_12 : addition : Addition Function Completed Successfully
INFO : main : logging_config_ex_12 : <module> : Addition of 10 & 20 is : 30.0

DEBUG : root :logging_config_ex_12 : addition : Inside Addition Function
WARNING : root :logging_config_ex_12 : addition : Warning : Parameter A is passed as String. Future versions won't support it.
INFO : root :logging_config_ex_12 : addition : Addition Function Completed Successfully
INFO : main : logging_config_ex_12 : <module> : Addition of '20' & 20 is : 40.0

DEBUG : root :logging_config_ex_12 : addition : Inside Addition Function
ERROR : root :logging_config_ex_12 : addition : Error Type : ValueError, Error Message : could not convert string to float: 'A'
INFO : main : logging_config_ex_12 : <module> : Addition of A & 20 is : None

This ends our small tutorial explaining how we can use 'logging.config' module configure logging module from 'dictionary' and 'configuration files'.

References

Sunny Solanki  Sunny Solanki

YouTube Subscribe Comfortable Learning through Video Tutorials?

If you are more comfortable learning through video tutorials then we would recommend that you subscribe to our YouTube channel.

Need Help Stuck Somewhere? Need Help with Coding? Have Doubts About the Topic/Code?

When going through coding examples, it's quite common to have doubts and errors.

If you have doubts about some code examples or are stuck somewhere when trying our code, send us an email at coderzcolumn07@gmail.com. We'll help you or point you in the direction where you can find a solution to your problem.

You can even send us a mail if you are trying something new and need guidance regarding coding. We'll try to respond as soon as possible.

Share Views Want to Share Your Views? Have Any Suggestions?

If you want to

  • provide some suggestions on topic
  • share your views
  • include some details in tutorial
  • suggest some new topics on which we should create tutorials/blogs
Please feel free to contact us at coderzcolumn07@gmail.com. We appreciate and value your feedbacks. You can also support us with a small contribution by clicking DONATE.


Subscribe to Our YouTube Channel

YouTube SubScribe

Newsletter Subscription