Updated On : Aug-13,2022 Time Investment : ~30 mins

tracemalloc - How to Profile Memory Usage By Python Code?

> What is Memory Profiling?

Memory Profiling is the process of recording memory usage (space complexity) by various parts of python code. We can record memory usage taken by individual parts or whole Python script/program as well.

It is one type of profiling whereas another one is time profiling (time complexity) where we record time taken by Python code/script/program. We have listed some useful time profilers in References section at the end of tutorial.

> Which Python Libraries to Use for Profiling Memory Usage?

Till 'v3.4', Python default installation did not have any memory profilers available in it. The 'v3.4' introduced a new module named 'tracemalloc' offering functionality to record memory usage traces.

The tracemalloc module allows us to monitor memory allocations in the python code. It lets us take memory snapshots at a particular point, perform various statistics on snapshots as well as perform differences between two snapshots to check object allocation between two snapshots.

Apart from 'tracemalloc', Python has other memory profilers (memory_profiler, pympler, guppy/heapy, Scalene, etc) which needs to be installed separately using 'pip' or 'conda'.

> What You Can Learn From This Article?

As a part of this tutorial, we have explained how to use Python Module tracemalloc to profile memory usage by python code and perform various operations. We have thoroughly covered the usage of various classes, methods, and attributes available through the module with the help of various examples. We have covered whole API of tracemalloc module. Tracemalloc can be used to profile memory usage by individual parts of Python program as well as whole Python program/script.

Below, we have listed important sections of tutorial to give an overview of the material covered.

Important Sections Of Tutorial

  1. Steps to Use "tracemalloc" for Memory Profiling
  2. Simple Example Explaining How to Use "tracemalloc"
  3. Useful Methods and Attributes of "tracemalloc" Library
  4. How to Clear Memory Traces?
  5. Store Memory Snapshots to a File and Load it Again
  6. Filter Unwanted Traces
  7. Compare Memory Snapshots to Find Out Differences
  8. Remove "tracemalloc" Module Entries from Traces
  9. Filter Snapshots to Keep Entries of Specified File
  10. How are Classes of "tracemalloc" Module Organized?

1. Steps to Use "tracemalloc" for Memory Profiling

  1. Start Memory Tracer ('start()' Method)
  2. Take snapshots using 'take_snapshot()'.
    • Once tracer has started, we can take Snapshots wherever we want using 'take_snapshot()'.
    • The snapshot will have memory usage from tracer start till the line where 'take_snapshot()' was called.
  3. Print Statistics by calling 'statistics()' Method on Snapshot object.
  4. Stop memory tracer ('stop()' Method).

2. Simple Example Explaining How to Use "tracemalloc"

As a part of the first example, we'll simply explain how to start tracing memory using tracemalloc, take snapshots, and print traced output.


Below is a list of methods that we have explained as a part of this example.

  • tracemalloc.start() - This method is available from tracemalloc module calling which will start tracing of memory. It accepts an integer argument named nframe which mentions a number of frames to allocate per call. The default value is 1.
  • tracemalloc.take_snapshot() - This method is available from the tracemalloc module which takes a memory snapshot when it’s called and returns the Snapshot object.
  • Snapshot.statistics() - This method is available through the Snapshot object and takes as an input key using which to sort records of tracing. It returns an iterator that has a list of Statistic objects. The single Statistic object has information about a single traceback (single line of code generally) which includes a number of objects and size of objects recorded in that traceback. Below is a list of keys that this method takes as input.
    • filename - Sort traces according to file names
    • lineno - sort traces according to line no in the file.
    • traceback - Sort traces according to trace taken order.

Below we have introduced the above-mentioned method with a simple example.

We are starting tracing at the beginning and then creating three lists of integers. We have then taken a memory snapshot and printed a list of tracebacks collected by that snapshot from the start of the traceback.

CODE: tracemalloc_ex1.py

import tracemalloc

tracemalloc.start()

l1 = [i for i in range(10000)]
l2 = [i*i for i in range(10000)]
l3 = [i*i*i for i in range(10000)]

snapshot = tracemalloc.take_snapshot()

for stat in snapshot.statistics("lineno"):
    print(stat)

tracemalloc.stop()

OUTPUT

tracemalloc_ex1.py:7: size=394 KiB, count=9994, average=40 B
tracemalloc_ex1.py:6: size=358 KiB, count=9984, average=37 B
tracemalloc_ex1.py:5: size=352 KiB, count=9745, average=37 B

3. Useful Methods and Attributes of "tracemalloc" Library

As a part of our second example, we'll explain a few attributes and methods of tracemalloc and statistic objects.


Below is a list of methods that we have explained as a part of this example.

  • Statistic.traceback - The traceback attribute of statistic object returns an object of class Traceback. The Traceback has information about individual traces which can have more than one frame (Frame).
  • Traceback.format() - This method formats traceback as a list of lines that were traced for this traceback.
  • tracemalloc.get_traceback_limit() - It returns an integer specifying maximum number of frames stored in single traceback. We can specify how many maximum frames to record per traceback in tracemalloc.start() method.
  • tracemalloc.get_object_traceback() - This method takes as input individual object and returns traceback object specifying where traceback for this object was taken.
  • tracemalloc.get_traced_memory() - It returns tuple of two integer specifying current size and peak size of block traced in bytes.
  • tracemalloc.get_tracemalloc_memory() - It returns integer specifying memory used by tracemalloc module in bytes.
  • tracemalloc.is_tracing() - It returns boolean specifying whether tracing is happening or stopped.

Below we have explained the usage of the above-mentioned method through example.

CODE: tracemalloc_ex2.py

import tracemalloc
import numpy as np

tracemalloc.start(25)

l1 = [i for i in range(10000)]
l2 = [i*i for i in range(10000)]
l3 = [i*i*i for i in range(10000)]
l4 = np.random.randint(1,100, (1000,))

snapshot = tracemalloc.take_snapshot()

print("========== SNAPSHOT =============")
for stat in snapshot.statistics("lineno"):
    print(stat)
    print(stat.traceback.format())

print("\n=========== USEFUL METHODS ===========")
print("\nTraceback Limit : ", tracemalloc.get_traceback_limit(), " Frames")

print("\nAllocation Location for List l4 : ",  tracemalloc.get_object_traceback(l4))

print("\nTraced Memory (Current, Peak): ", tracemalloc.get_traced_memory())

#tracemalloc.reset_peak()

#print("\nTraced Memory : ", tracemalloc.get_traced_memory())

print("\nMemory Usage by tracemalloc Module : ", tracemalloc.get_tracemalloc_memory(), " bytes")

print("\nTracing Status : ",tracemalloc.is_tracing())

tracemalloc.stop()

OUTPUT

========== SNAPSHOT =============
tracemalloc_ex2.py:8: size=394 KiB, count=9994, average=40 B
['  File "tracemalloc_ex2.py", line 8', '    l3 = [i*i*i for i in range(10000)]']
tracemalloc_ex2.py:7: size=358 KiB, count=9984, average=37 B
['  File "tracemalloc_ex2.py", line 7', '    l2 = [i*i for i in range(10000)]']
tracemalloc_ex2.py:6: size=352 KiB, count=9744, average=37 B
['  File "tracemalloc_ex2.py", line 6', '    l1 = [i for i in range(10000)]']
tracemalloc_ex2.py:9: size=8138 B, count=3, average=2713 B
['  File "tracemalloc_ex2.py", line 9', '    l4 = np.random.randint(1,100, (1000,))']
<__array_function__ internals>:6: size=536 B, count=1, average=536 B
['  File "<__array_function__ internals>", line 6']
/home/sunny/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2911: size=528 B, count=1, average=528 B
['  File "/home/sunny/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 2911', '    keepdims=keepdims, initial=initial, where=where)']
/home/sunny/anaconda3/lib/python3.7/site-packages/numpy/core/_dtype.py:334: size=488 B, count=1, average=488 B
['  File "/home/sunny/anaconda3/lib/python3.7/site-packages/numpy/core/_dtype.py", line 334', '    if np.issubdtype(dtype, np.flexible) and not _isunsized(dtype):']
<__array_function__ internals>:4: size=464 B, count=1, average=464 B
['  File "<__array_function__ internals>", line 4']
/home/sunny/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:74: size=448 B, count=1, average=448 B
['  File "/home/sunny/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 74', '    passkwargs = {k: v for k, v in kwargs.items()']
tracemalloc_ex2.py:11: size=432 B, count=1, average=432 B
['  File "tracemalloc_ex2.py", line 11', '    snapshot = tracemalloc.take_snapshot()']
/home/sunny/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:90: size=145 B, count=2, average=72 B
['  File "/home/sunny/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 90', '    return ufunc.reduce(obj, axis, dtype, out, **passkwargs)']

=========== USEFUL METHODS ===========

Traceback Limit :  25  Frames

Allocation Location for List l4 :  tracemalloc_ex2.py:9

Traced Memory (Current, Peak):  (1472281, 1487679)

Memory Usage by tracemalloc Module :  2504012  bytes

Tracing Status :  True

4. How to Clear Memory Traces?

As a part of this example, we have simply explained how we can start/stop tracemalloc and clear traces.


Below is a list of methods that we have explained as a part of this example.

  • tracemalloc.stop() - This method stops tracemalloc from taking traces.
  • tracemalloc.clear_traces() - This method clears all traces taken from the beginning of the start() method.

Please make a NOTE that traces info would still be available for snapshot objects which were taken but it won't be available if you call the take_snapshot() method after this method.


Below we have first taken a snapshot after creating three lists of integers.

We have then created another list and taken snapshots again. We have then cleared all traces which are traces of 4 lists created since the beginning.

We have then created the fifth list and taken a snapshot again.

Please make a note from the output that the 3rd snapshot has information only about the fifth list created and all traces before it is cleared.

We have also explained the usage of stop() at the end.

CODE: tracemalloc_ex3.py

import tracemalloc

tracemalloc.start()

l1 = [i for i in range(1000000)]
l2 = [i*i for i in range(10000)]
l3 = [i*i*i for i in range(10000)]

snapshot1 = tracemalloc.take_snapshot()

print("================ SNAPSHOT 1 =================")
for stat in snapshot1.statistics("lineno"):
    print(stat)

l4 = [i*i*i*i for i in range(10000)]



snapshot2 = tracemalloc.take_snapshot()

print("\n================ SNAPSHOT 2 =================")
for stat in snapshot2.statistics("lineno"):
    print(stat)

tracemalloc.clear_traces()

l5 = [i*2 for i in range(10000)]

snapshot3 = tracemalloc.take_snapshot()

print("\n================ SNAPSHOT 3 =================")
for stat in snapshot3.statistics("lineno"):
    print(stat)


print("\nTracing Status : ", tracemalloc.is_tracing())

tracemalloc.stop()

print("\nTracing Status : ", tracemalloc.is_tracing())

try:
    print("\nTrying to Take Snapshot After Tracing is Stopped.")
    snap = tracemalloc.take_snapshot()
except Exception as e:
    print("Exception : ", e)

OUTPUT

================ SNAPSHOT 1 =================
tracemalloc_ex3.py:5: size=35.0 MiB, count=999745, average=37 B
tracemalloc_ex3.py:7: size=394 KiB, count=9994, average=40 B
tracemalloc_ex3.py:6: size=358 KiB, count=9984, average=37 B

================ SNAPSHOT 2 =================
tracemalloc_ex3.py:5: size=35.0 MiB, count=999745, average=37 B
tracemalloc_ex3.py:15: size=433 KiB, count=9997, average=44 B
tracemalloc_ex3.py:7: size=394 KiB, count=9994, average=40 B
tracemalloc_ex3.py:6: size=358 KiB, count=9984, average=37 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:209: size=864 B, count=2, average=432 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:165: size=832 B, count=2, average=416 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:397: size=656 B, count=5, average=131 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:479: size=576 B, count=3, average=192 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:507: size=576 B, count=1, average=576 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:534: size=552 B, count=3, average=184 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:472: size=480 B, count=2, average=240 B
tracemalloc_ex3.py:12: size=464 B, count=1, average=464 B
tracemalloc_ex3.py:13: size=456 B, count=1, average=456 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:53: size=448 B, count=1, average=448 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:509: size=424 B, count=1, average=424 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:194: size=424 B, count=1, average=424 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:54: size=424 B, count=1, average=424 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:475: size=416 B, count=1, average=416 B
tracemalloc_ex3.py:11: size=314 B, count=5, average=63 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:459: size=240 B, count=2, average=120 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:458: size=240 B, count=2, average=120 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:185: size=168 B, count=3, average=56 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:65: size=144 B, count=2, average=72 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:291: size=112 B, count=2, average=56 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:508: size=64 B, count=1, average=64 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:289: size=56 B, count=1, average=56 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:180: size=56 B, count=1, average=56 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:477: size=28 B, count=1, average=28 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:476: size=28 B, count=1, average=28 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:27: size=24 B, count=1, average=24 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:15: size=24 B, count=1, average=24 B

================ SNAPSHOT 3 =================
tracemalloc_ex3.py:27: size=356 KiB, count=9873, average=37 B

Tracing Status :  True

Tracing Status :  False

Trying to Take Snapshot After Tracing is Stopped.
Exception :  the tracemalloc module must be tracing memory allocations to take a snapshot

5. Store Memory Snapshots to a File and Load it Again

As a part of our fourth example, we'll explain how we can store a snapshot of traces into a file on disk and then load it again.


Below is a list of methods that we have explained as a part of this example.

  • Snapshot.dump() - This method takes as input filename to which store traces information.
  • Snapshot.load() - This method takes as input filename from which load traces information.

Below we have explained the usage of both methods.

CODE: tracemalloc_ex4.py

import tracemalloc

tracemalloc.start()

l1 = [i for i in range(10000)]
l2 = [i*i for i in range(10000)]
l3 = [i*i*i for i in range(10000)]

snapshot = tracemalloc.take_snapshot()

for stat in snapshot.statistics("lineno"):
    print(stat)

snapshot.dump("snap.out")

snapshot_loaded = tracemalloc.Snapshot.load("snap.out")

print("\nLoaded Snapshot From File :")
for stat in snapshot_loaded.statistics("lineno"):
    print(stat)

tracemalloc.stop()

OUTPUT

tracemalloc_ex4.py:7: size=394 KiB, count=9994, average=40 B
tracemalloc_ex4.py:6: size=358 KiB, count=9984, average=37 B
tracemalloc_ex4.py:5: size=352 KiB, count=9745, average=37 B

Loaded Snapshot From File :
tracemalloc_ex4.py:7: size=394 KiB, count=9994, average=40 B
tracemalloc_ex4.py:6: size=358 KiB, count=9984, average=37 B
tracemalloc_ex4.py:5: size=352 KiB, count=9745, average=37 B

6. Filter Unwanted Traces

As a part of this example, we'll explain how we can filter out traces from a list of all traces recorded by the tracemalloc. There are two main classes provided by tracemalloc for filtering traces.


  1. DomainFilter - This class lets us filter traces by address space of objects. It takes two parameters as input.

    • domain - It accepts integers specifying address space of memory block. The integer 0 is generally used by python memory traces but the number will be different for C.
    • inclusive - It accepts boolean specifying whether to include or exclude traces specified by the domain attribute.
  2. Filter - This class lets filter traces based on combinations of the filename, line number, and domain. It has the below-mentioned parameters.

    • domain - It accepts integers specifying address space of memory block. The integer 0 is generally used by python memory traces but the number will be different for C.
    • filename_pattern - It accepts string specifying pattern for which filenames will be matched in traces and based on the inclusive flag they will be included or excluded.
    • lineno - It accepts integer specifying traces with that line numbers to include.
    • inclusive - It accepts boolean specifying whether to include or exclude traces specified by domain, filename_pattern, and lineno attributes.
    • all_frames - It accepts a boolean flag specifying whether to include all frames per trace or not.

We have explained the usage of filters through the examples below.

The first snapshot includes all traces for memory blocks used by python.

The second snapshot includes traces of memory blocks that are not created by python. It can be due to C as numpy is built on it.

The third snapshot includes entries where the filename is tracemalloc_ex5.py and the fourth snapshot excludes entries with that file name.

CODE: tracemalloc_ex5.py

import tracemalloc
import numpy as np

tracemalloc.start(10)

l2 = [i*i for i in range(10000)]
l1 = [i for i in range(10000)]
l3 = [i*i*i for i in range(10000)]
l4 = [i*2 for i in range(10000)]
l5 = np.random.randint(1,100, (1000,))

snapshot = tracemalloc.take_snapshot()

print("========== Original Snapshot ===========")
for stat in snapshot.statistics("lineno"):
    print(stat)

print("\n========= Filtered Snapshot 1 =============")
filtr = tracemalloc.DomainFilter(inclusive=True, domain=0)
snap_shot = snapshot.filter_traces(filters=[filtr])

for stat in snap_shot.statistics("lineno"):
    print(stat)


filtr = tracemalloc.DomainFilter(inclusive=False, domain=0)
snap_shot = snapshot.filter_traces(filters=[filtr])

print("\n========= Filtered Snapshot 2 =============")
for stat in snap_shot.statistics("lineno"):
    print(stat)

filtr = tracemalloc.Filter(inclusive=True, filename_pattern="tracemalloc_ex5.py")
snap_shot = snapshot.filter_traces([filtr])

print("\n========= Filtered Snapshot 3 =============")
for stat in snap_shot.statistics("lineno"):
    print(stat)


filtr = tracemalloc.Filter(inclusive=False, filename_pattern="tracemalloc_ex5.py")
snap_shot = snapshot.filter_traces([filtr])

print("\n========= Filtered Snapshot 4 =============")
for stat in snap_shot.statistics("lineno"):
    print(stat)

tracemalloc.stop()

OUTPUT

========== Original Snapshot ===========
tracemalloc_ex5.py:8: size=394 KiB, count=9994, average=40 B
tracemalloc_ex5.py:6: size=358 KiB, count=9984, average=37 B
tracemalloc_ex5.py:9: size=355 KiB, count=9872, average=37 B
tracemalloc_ex5.py:7: size=352 KiB, count=9744, average=37 B
tracemalloc_ex5.py:10: size=9050 B, count=5, average=1810 B
<__array_function__ internals>:6: size=536 B, count=1, average=536 B
/home/sunny/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2911: size=528 B, count=1, average=528 B
<__array_function__ internals>:4: size=464 B, count=1, average=464 B
/home/sunny/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:74: size=448 B, count=1, average=448 B
tracemalloc_ex5.py:12: size=432 B, count=1, average=432 B
/home/sunny/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:90: size=145 B, count=2, average=72 B

========= Filtered Snapshot 1 =============
tracemalloc_ex5.py:8: size=394 KiB, count=9994, average=40 B
tracemalloc_ex5.py:6: size=358 KiB, count=9984, average=37 B
tracemalloc_ex5.py:9: size=355 KiB, count=9872, average=37 B
tracemalloc_ex5.py:7: size=352 KiB, count=9744, average=37 B
tracemalloc_ex5.py:10: size=1050 B, count=4, average=262 B
<__array_function__ internals>:6: size=536 B, count=1, average=536 B
/home/sunny/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2911: size=528 B, count=1, average=528 B
<__array_function__ internals>:4: size=464 B, count=1, average=464 B
/home/sunny/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:74: size=448 B, count=1, average=448 B
tracemalloc_ex5.py:12: size=432 B, count=1, average=432 B
/home/sunny/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:90: size=145 B, count=2, average=72 B

========= Filtered Snapshot 2 =============
tracemalloc_ex5.py:10: size=8000 B, count=1, average=8000 B

========= Filtered Snapshot 3 =============
tracemalloc_ex5.py:8: size=394 KiB, count=9994, average=40 B
tracemalloc_ex5.py:6: size=358 KiB, count=9984, average=37 B
tracemalloc_ex5.py:9: size=355 KiB, count=9872, average=37 B
tracemalloc_ex5.py:7: size=352 KiB, count=9744, average=37 B
tracemalloc_ex5.py:10: size=9050 B, count=5, average=1810 B
tracemalloc_ex5.py:12: size=432 B, count=1, average=432 B

========= Filtered Snapshot 4 =============
<__array_function__ internals>:6: size=536 B, count=1, average=536 B
/home/sunny/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2911: size=528 B, count=1, average=528 B
<__array_function__ internals>:4: size=464 B, count=1, average=464 B
/home/sunny/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:74: size=448 B, count=1, average=448 B
/home/sunny/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:90: size=145 B, count=2, average=72 B

7. Compare Memory Snapshots to Find Out Differences

As a part of this example, we'll explain how we can compare two snapshots and find out the difference in traces between them.


Below is a list of methods that we have explained as a part of this example.

  • Snapshot.compare_to() - This method can be called on any snapshot object passing it another snapshot object to get the difference between two snapshots. It also takes as input the key_type parameter which will be used as the primary key to find out the difference between two snapshots. It'll return an object of class StatisticDiff. This object has information about differences.

Below we have explained through example, how we can get the difference between two snapshots. Please make a note that different traces taken before the first snapshot are not present.

CODE: tracemalloc_ex6.py

import tracemalloc

tracemalloc.start()

l1 = [i for i in range(10000)]
l2 = [i*i for i in range(10000)]
l3 = [i*i*i for i in range(10000)]

snapshot1 = tracemalloc.take_snapshot()

print("SNAPSHOT - 1")
for stat in snapshot1.statistics("lineno"):
    print(stat)


l4 = [i*i*i*i for i in range(10000)]
l5 = [i*i*i*i*i for i in range(10000)]

print("\nSNAPSHOT - 2")
snapshot2 = tracemalloc.take_snapshot()

for stat in snapshot2.statistics("lineno"):
    #if "tracemalloc_ex6.py" in str(stat):
    print(stat)

print("\nDIFFERENCE")
for diff in snapshot2.compare_to(snapshot1,"lineno"):
    #if "tracemalloc_ex6.py" in str(diff):
    print(diff)

tracemalloc.stop()

OUTPUT

SNAPSHOT - 1
tracemalloc_ex6.py:7: size=394 KiB, count=9994, average=40 B
tracemalloc_ex6.py:6: size=358 KiB, count=9984, average=37 B
tracemalloc_ex6.py:5: size=352 KiB, count=9745, average=37 B

SNAPSHOT - 2
tracemalloc_ex6.py:17: size=436 KiB, count=9998, average=45 B
tracemalloc_ex6.py:16: size=433 KiB, count=9997, average=44 B
tracemalloc_ex6.py:7: size=394 KiB, count=9994, average=40 B
tracemalloc_ex6.py:6: size=358 KiB, count=9984, average=37 B
tracemalloc_ex6.py:5: size=352 KiB, count=9745, average=37 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:209: size=864 B, count=2, average=432 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:165: size=832 B, count=2, average=416 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:479: size=576 B, count=3, average=192 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:507: size=576 B, count=1, average=576 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:397: size=528 B, count=3, average=176 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:472: size=480 B, count=2, average=240 B
tracemalloc_ex6.py:12: size=464 B, count=1, average=464 B
tracemalloc_ex6.py:13: size=456 B, count=1, average=456 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:53: size=448 B, count=1, average=448 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:509: size=424 B, count=1, average=424 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:194: size=424 B, count=1, average=424 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:54: size=424 B, count=1, average=424 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:475: size=416 B, count=1, average=416 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:65: size=144 B, count=2, average=72 B
tracemalloc_ex6.py:11: size=106 B, count=2, average=53 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:508: size=64 B, count=1, average=64 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:534: size=56 B, count=1, average=56 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:185: size=56 B, count=1, average=56 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:291: size=40 B, count=1, average=40 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:477: size=28 B, count=1, average=28 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:476: size=28 B, count=1, average=28 B

DIFFERENCE
tracemalloc_ex6.py:17: size=436 KiB (+436 KiB), count=9998 (+9998), average=45 B
tracemalloc_ex6.py:16: size=433 KiB (+433 KiB), count=9997 (+9997), average=44 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:209: size=864 B (+864 B), count=2 (+2), average=432 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:165: size=832 B (+832 B), count=2 (+2), average=416 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:479: size=576 B (+576 B), count=3 (+3), average=192 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:507: size=576 B (+576 B), count=1 (+1), average=576 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:397: size=528 B (+528 B), count=3 (+3), average=176 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:472: size=480 B (+480 B), count=2 (+2), average=240 B
tracemalloc_ex6.py:12: size=464 B (+464 B), count=1 (+1), average=464 B
tracemalloc_ex6.py:13: size=456 B (+456 B), count=1 (+1), average=456 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:53: size=448 B (+448 B), count=1 (+1), average=448 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:509: size=424 B (+424 B), count=1 (+1), average=424 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:194: size=424 B (+424 B), count=1 (+1), average=424 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:54: size=424 B (+424 B), count=1 (+1), average=424 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:475: size=416 B (+416 B), count=1 (+1), average=416 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:65: size=144 B (+144 B), count=2 (+2), average=72 B
tracemalloc_ex6.py:11: size=106 B (+106 B), count=2 (+2), average=53 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:508: size=64 B (+64 B), count=1 (+1), average=64 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:534: size=56 B (+56 B), count=1 (+1), average=56 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:185: size=56 B (+56 B), count=1 (+1), average=56 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:291: size=40 B (+40 B), count=1 (+1), average=40 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:477: size=28 B (+28 B), count=1 (+1), average=28 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:476: size=28 B (+28 B), count=1 (+1), average=28 B
tracemalloc_ex6.py:7: size=394 KiB (+0 B), count=9994 (+0), average=40 B
tracemalloc_ex6.py:6: size=358 KiB (+0 B), count=9984 (+0), average=37 B
tracemalloc_ex6.py:5: size=352 KiB (+0 B), count=9745 (+0), average=37 B

8. Remove "tracemalloc" Module Entries from Traces

Our seventh example is exactly the same as our sixth example with the only difference that we have used a filter to remove entries related to the tracemalloc module itself.

We have specified tracemalloc filename (tracemalloc.file) to filename_pattern of Filter class with the inclusive attribute as False to exclude entries with tracemalloc module.

CODE: tracemalloc_ex7.py

import tracemalloc

tracemalloc.start()

l1 = [i for i in range(10000)]
l2 = [i*i for i in range(10000)]
l3 = [i*i*i for i in range(10000)]

snapshot1 = tracemalloc.take_snapshot()

print("SNAPSHOT - 1")
for stat in snapshot1.statistics("lineno"):
    print(stat)


l4 = [i*i*i*i for i in range(10000)]
l5 = [i*i*i*i*i for i in range(10000)]

print("\nSNAPSHOT - 2")
snapshot2 = tracemalloc.take_snapshot()
filtr = tracemalloc.Filter(inclusive=False, filename_pattern=tracemalloc.__file__)
snapshot2 = snapshot2.filter_traces([filtr])

for stat in snapshot2.statistics("lineno"):
        print(stat)

print("\nDIFFERENCE")
for diff in snapshot2.compare_to(snapshot1,"lineno"):
        print(diff)

tracemalloc.stop()

OUTPUT

SNAPSHOT - 1
tracemalloc_ex7.py:7: size=394 KiB, count=9994, average=40 B
tracemalloc_ex7.py:6: size=358 KiB, count=9984, average=37 B
tracemalloc_ex7.py:5: size=352 KiB, count=9745, average=37 B

SNAPSHOT - 2
tracemalloc_ex7.py:17: size=436 KiB, count=9998, average=45 B
tracemalloc_ex7.py:16: size=433 KiB, count=9997, average=44 B
tracemalloc_ex7.py:7: size=394 KiB, count=9994, average=40 B
tracemalloc_ex7.py:6: size=358 KiB, count=9984, average=37 B
tracemalloc_ex7.py:5: size=352 KiB, count=9745, average=37 B
tracemalloc_ex7.py:12: size=464 B, count=1, average=464 B
tracemalloc_ex7.py:13: size=456 B, count=1, average=456 B
tracemalloc_ex7.py:11: size=106 B, count=2, average=53 B

DIFFERENCE
tracemalloc_ex7.py:17: size=436 KiB (+436 KiB), count=9998 (+9998), average=45 B
tracemalloc_ex7.py:16: size=433 KiB (+433 KiB), count=9997 (+9997), average=44 B
tracemalloc_ex7.py:12: size=464 B (+464 B), count=1 (+1), average=464 B
tracemalloc_ex7.py:13: size=456 B (+456 B), count=1 (+1), average=456 B
tracemalloc_ex7.py:11: size=106 B (+106 B), count=2 (+2), average=53 B
tracemalloc_ex7.py:7: size=394 KiB (+0 B), count=9994 (+0), average=40 B
tracemalloc_ex7.py:6: size=358 KiB (+0 B), count=9984 (+0), average=37 B
tracemalloc_ex7.py:5: size=352 KiB (+0 B), count=9745 (+0), average=37 B

9. Filter Snapshots to Keep Entries of Specified File

Our eighth example is also exactly the same as the seventh example but we have specified in a different way how we can include only entries pertaining to the file which we are tracing.

We have specified tracemalloc_ex8.py as a filename pattern to the Filter object. This way it'll eliminate entries of tracemalloc module and only keep the entry of this file.

CODE: tracemalloc_ex8.py

import tracemalloc

tracemalloc.start()

l1 = [i for i in range(10000)]
l2 = [i*i for i in range(10000)]
l3 = [i*i*i for i in range(10000)]

snapshot1 = tracemalloc.take_snapshot()

print("SNAPSHOT - 1")
for stat in snapshot1.statistics("lineno"):
    print(stat)


l4 = [i*i*i*i for i in range(10000)]
l5 = [i*i*i*i*i for i in range(10000)]

print("\nSNAPSHOT - 2")
snapshot2 = tracemalloc.take_snapshot()

filtr = tracemalloc.Filter(inclusive=True, filename_pattern="tracemalloc_ex8.py")
snapshot2 = snapshot2.filter_traces([filtr])

for stat in snapshot2.statistics("lineno"):
    print(stat)

print("\nDIFFERENCE")
for diff in snapshot2.compare_to(snapshot1,"lineno"):
    print(diff)

tracemalloc.stop()

OUTPUT

SNAPSHOT - 1
tracemalloc_ex8.py:7: size=394 KiB, count=9994, average=40 B
tracemalloc_ex8.py:6: size=358 KiB, count=9984, average=37 B
tracemalloc_ex8.py:5: size=352 KiB, count=9745, average=37 B

SNAPSHOT - 2
tracemalloc_ex8.py:17: size=436 KiB, count=9998, average=45 B
tracemalloc_ex8.py:16: size=433 KiB, count=9997, average=44 B
tracemalloc_ex8.py:7: size=394 KiB, count=9994, average=40 B
tracemalloc_ex8.py:6: size=358 KiB, count=9984, average=37 B
tracemalloc_ex8.py:5: size=352 KiB, count=9745, average=37 B
tracemalloc_ex8.py:12: size=464 B, count=1, average=464 B
tracemalloc_ex8.py:13: size=456 B, count=1, average=456 B
tracemalloc_ex8.py:11: size=106 B, count=2, average=53 B

DIFFERENCE
tracemalloc_ex8.py:17: size=436 KiB (+436 KiB), count=9998 (+9998), average=45 B
tracemalloc_ex8.py:16: size=433 KiB (+433 KiB), count=9997 (+9997), average=44 B
tracemalloc_ex8.py:12: size=464 B (+464 B), count=1 (+1), average=464 B
tracemalloc_ex8.py:13: size=456 B (+456 B), count=1 (+1), average=456 B
tracemalloc_ex8.py:11: size=106 B (+106 B), count=2 (+2), average=53 B
tracemalloc_ex8.py:7: size=394 KiB (+0 B), count=9994 (+0), average=40 B
tracemalloc_ex8.py:6: size=358 KiB (+0 B), count=9984 (+0), average=37 B
tracemalloc_ex8.py:5: size=352 KiB (+0 B), count=9745 (+0), average=37 B

10. How are Classes of "tracemalloc" Module Organized?

  1. Snapshot Object has list of Trace objects recording each trace of memory.
  2. Trace object has Traceback object with tracing information.
  3. Traceback object has recorded trace as a list of Frame objects.
  4. Statistic object has stats of one Trace.
  5. Filter objects let us filter traces from Snapshot object based on various conditions.
  6. StatisticDiff object has a difference in memory traces between two Snapshot objects.
Sunny Solanki  Sunny Solanki

Share Views Stuck Somewhere? Need Help with Coding? Have Doubts About the Topic/Code?

When going through coding examples, it's quite common to have doubts and errors.

If you have doubts about some code examples or are stuck somewhere when trying our code, send us an email at coderzcolumn07@gmail.com. We'll help you or point you in the direction where you can find a solution to your problem.

You can even send us a mail if you are trying something new and need guidance regarding coding. We'll try to respond as soon as possible.

Share Views Want to Share Your Views? Have Any Suggestions?

If you want to

  • provide some suggestions on topic
  • share your views
  • include some details in tutorial
  • suggest some new topics on which we should create tutorials/blogs
Please feel free to contact us at coderzcolumn07@gmail.com. We appreciate and value your feedbacks. You can also support us with a small contribution by clicking DONATE.