The tracemalloc module which became available from Python version 3.4. The tracemalloc module allows us to monitor memory allocations in the python code. It lets us take memory snapshots at a particular point, perform various statistics on snapshots as well as perform difference between two snapshots to check object allocation between two snapshots.
As a part of this tutorial, we'll explain how we can use tracemalloc API to trace memory usage in python code and perform various operations. We'll be explaining the usage of various classes, methods, and attributes available through the module with the help of various examples.
If you are interested in learning about other python profilers then please feel free to check our references section which has a list of tutorials on python profilers.
As a part of the first example, we'll simply explain how to start tracing memory using tracemalloc, take snapshots, and print traced output.
Below is a list of methods which we'll explain as a part of this example.
nframe
which mentions a number of frames to allocate per call. The default value is 1.Snapshot
object.Snapshot
object and takes as input key using which to sort records of tracing. It returns an iterator that has a list of Statistic
objects. The single Statistic
object has information about a single traceback (single line of code generally) which includes a number of objects and size of objects recorded in that traceback. Below is a list of keys that this method takes as input.filename
- Sort traces according to file nameslineno
- sort traces according to line no in the file.traceback
- Sort traces according to trace taken order.Below we have introduced the above-mentioned method with a simple example. We are starting tracing at the beginning and then creating three lists of integers. We have then taken a memory snapshot and printed a list of tracebacks collected by that snapshot from the starting of the traceback.
CODE
import tracemalloc
tracemalloc.start()
l1 = [i for i in range(10000)]
l2 = [i*i for i in range(10000)]
l3 = [i*i*i for i in range(10000)]
snapshot = tracemalloc.take_snapshot()
for stat in snapshot.statistics("lineno"):
print(stat)
OUTPUT
tracemalloc_ex1.py:7: size=394 KiB, count=9994, average=40 B
tracemalloc_ex1.py:6: size=358 KiB, count=9984, average=37 B
tracemalloc_ex1.py:5: size=352 KiB, count=9745, average=37 B
As a part of our second example, we'll explain a few attributes and methods of tracemalloc and statistic objects.
Traceback
. The Traceback
has information about individual trace which can have more than one frame (Frame
).tracemalloc.start()
method.Below we have explained the usage of the above-mentioned method through example.
CODE
import tracemalloc
import numpy as np
tracemalloc.start(25)
l1 = [i for i in range(10000)]
l2 = [i*i for i in range(10000)]
l3 = [i*i*i for i in range(10000)]
l4 = np.random.randint(1,100, (1000,))
snapshot = tracemalloc.take_snapshot()
print("========== SNAPSHOT =============")
for stat in snapshot.statistics("lineno"):
print(stat)
print(stat.traceback.format())
print("\n=========== USEFUL METHODS ===========")
print("\nTraceback Limit : ", tracemalloc.get_traceback_limit(), " Frames")
print("\nAllocation Location for List l4 : ", tracemalloc.get_object_traceback(l4))
print("\nTraced Memory (Current, Peak): ", tracemalloc.get_traced_memory())
#tracemalloc.reset_peak()
#print("\nTraced Memory : ", tracemalloc.get_traced_memory())
print("\nMemory Usage by tracemalloc Module : ", tracemalloc.get_tracemalloc_memory(), " bytes")
print("\nTracing Status : ",tracemalloc.is_tracing())
OUTPUT
========== SNAPSHOT =============
tracemalloc_ex2.py:8: size=394 KiB, count=9994, average=40 B
[' File "tracemalloc_ex2.py", line 8', ' l3 = [i*i*i for i in range(10000)]']
tracemalloc_ex2.py:7: size=358 KiB, count=9984, average=37 B
[' File "tracemalloc_ex2.py", line 7', ' l2 = [i*i for i in range(10000)]']
tracemalloc_ex2.py:6: size=352 KiB, count=9744, average=37 B
[' File "tracemalloc_ex2.py", line 6', ' l1 = [i for i in range(10000)]']
tracemalloc_ex2.py:9: size=8138 B, count=3, average=2713 B
[' File "tracemalloc_ex2.py", line 9', ' l4 = np.random.randint(1,100, (1000,))']
<__array_function__ internals>:6: size=536 B, count=1, average=536 B
[' File "<__array_function__ internals>", line 6']
/home/sunny/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2911: size=528 B, count=1, average=528 B
[' File "/home/sunny/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 2911', ' keepdims=keepdims, initial=initial, where=where)']
/home/sunny/anaconda3/lib/python3.7/site-packages/numpy/core/_dtype.py:334: size=488 B, count=1, average=488 B
[' File "/home/sunny/anaconda3/lib/python3.7/site-packages/numpy/core/_dtype.py", line 334', ' if np.issubdtype(dtype, np.flexible) and not _isunsized(dtype):']
<__array_function__ internals>:4: size=464 B, count=1, average=464 B
[' File "<__array_function__ internals>", line 4']
/home/sunny/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:74: size=448 B, count=1, average=448 B
[' File "/home/sunny/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 74', ' passkwargs = {k: v for k, v in kwargs.items()']
tracemalloc_ex2.py:11: size=432 B, count=1, average=432 B
[' File "tracemalloc_ex2.py", line 11', ' snapshot = tracemalloc.take_snapshot()']
/home/sunny/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:90: size=145 B, count=2, average=72 B
[' File "/home/sunny/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 90', ' return ufunc.reduce(obj, axis, dtype, out, **passkwargs)']
=========== USEFUL METHODS ===========
Traceback Limit : 25 Frames
Allocation Location for List l4 : tracemalloc_ex2.py:9
Traced Memory (Current, Peak): (1472281, 1487679)
Memory Usage by tracemalloc Module : 2504012 bytes
Tracing Status : True
As a part of this example, we have simply explained how we can start/stop tracemalloc and clear traces.
start()
method. Please make a note that traces info would still be available for snapshot objects which were taken but it won't be available if you call the take_snapshot()
method after this method.Below we have first taken a snapshot after creating three lists of integers. We have then created another list and taken snapshots again. We have then cleared all traces which are traces of 4 lists created since the beginning. We have then created the fifth list and taken snapshot again. Please make a note from the output that the 3rd snapshot has information only about the fifth list created and all traces before it is cleared. We have also explained the usage of stop()
at the end.
CODE
import tracemalloc
tracemalloc.start()
l1 = [i for i in range(1000000)]
l2 = [i*i for i in range(10000)]
l3 = [i*i*i for i in range(10000)]
snapshot1 = tracemalloc.take_snapshot()
print("================ SNAPSHOT 1 =================")
for stat in snapshot1.statistics("lineno"):
print(stat)
l4 = [i*i*i*i for i in range(10000)]
snapshot2 = tracemalloc.take_snapshot()
print("\n================ SNAPSHOT 2 =================")
for stat in snapshot2.statistics("lineno"):
print(stat)
tracemalloc.clear_traces()
l5 = [i*2 for i in range(10000)]
snapshot3 = tracemalloc.take_snapshot()
print("\n================ SNAPSHOT 3 =================")
for stat in snapshot3.statistics("lineno"):
print(stat)
print("\nTracing Status : ", tracemalloc.is_tracing())
tracemalloc.stop()
print("\nTracing Status : ", tracemalloc.is_tracing())
try:
print("\nTrying to Take Snapshot After Tracing is Stopped.")
snap = tracemalloc.take_snapshot()
except Exception as e:
print("Exception : ", e)
OUTPUT
================ SNAPSHOT 1 =================
tracemalloc_ex3.py:5: size=35.0 MiB, count=999745, average=37 B
tracemalloc_ex3.py:7: size=394 KiB, count=9994, average=40 B
tracemalloc_ex3.py:6: size=358 KiB, count=9984, average=37 B
================ SNAPSHOT 2 =================
tracemalloc_ex3.py:5: size=35.0 MiB, count=999745, average=37 B
tracemalloc_ex3.py:15: size=433 KiB, count=9997, average=44 B
tracemalloc_ex3.py:7: size=394 KiB, count=9994, average=40 B
tracemalloc_ex3.py:6: size=358 KiB, count=9984, average=37 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:209: size=864 B, count=2, average=432 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:165: size=832 B, count=2, average=416 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:397: size=656 B, count=5, average=131 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:479: size=576 B, count=3, average=192 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:507: size=576 B, count=1, average=576 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:534: size=552 B, count=3, average=184 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:472: size=480 B, count=2, average=240 B
tracemalloc_ex3.py:12: size=464 B, count=1, average=464 B
tracemalloc_ex3.py:13: size=456 B, count=1, average=456 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:53: size=448 B, count=1, average=448 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:509: size=424 B, count=1, average=424 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:194: size=424 B, count=1, average=424 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:54: size=424 B, count=1, average=424 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:475: size=416 B, count=1, average=416 B
tracemalloc_ex3.py:11: size=314 B, count=5, average=63 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:459: size=240 B, count=2, average=120 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:458: size=240 B, count=2, average=120 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:185: size=168 B, count=3, average=56 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:65: size=144 B, count=2, average=72 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:291: size=112 B, count=2, average=56 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:508: size=64 B, count=1, average=64 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:289: size=56 B, count=1, average=56 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:180: size=56 B, count=1, average=56 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:477: size=28 B, count=1, average=28 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:476: size=28 B, count=1, average=28 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:27: size=24 B, count=1, average=24 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:15: size=24 B, count=1, average=24 B
================ SNAPSHOT 3 =================
tracemalloc_ex3.py:27: size=356 KiB, count=9873, average=37 B
Tracing Status : True
Tracing Status : False
Trying to Take Snapshot After Tracing is Stopped.
Exception : the tracemalloc module must be tracing memory allocations to take a snapshot
As a part of our fourth example, we'll explain how we can store a snapshot of traces into a file on disk and then load it again.
Below we have explained the usage of both methods.
CODE
import tracemalloc
tracemalloc.start()
l1 = [i for i in range(10000)]
l2 = [i*i for i in range(10000)]
l3 = [i*i*i for i in range(10000)]
snapshot = tracemalloc.take_snapshot()
for stat in snapshot.statistics("lineno"):
print(stat)
snapshot.dump("snap.out")
snapshot_loaded = tracemalloc.Snapshot.load("snap.out")
print("\nLoaded Snapshot From File :")
for stat in snapshot_loaded.statistics("lineno"):
print(stat)
OUTPUT
tracemalloc_ex4.py:7: size=394 KiB, count=9994, average=40 B
tracemalloc_ex4.py:6: size=358 KiB, count=9984, average=37 B
tracemalloc_ex4.py:5: size=352 KiB, count=9745, average=37 B
Loaded Snapshot From File :
tracemalloc_ex4.py:7: size=394 KiB, count=9994, average=40 B
tracemalloc_ex4.py:6: size=358 KiB, count=9984, average=37 B
tracemalloc_ex4.py:5: size=352 KiB, count=9745, average=37 B
As a part of this example, we'll explain how we can filter out traces from a list of all traces recorded by the tracemalloc. There are two main classes provided by tracemalloc for filtering traces.
DomainFilter - This class lets us filter traces by address space of objects. It takes two parameters as input.
domain
attribute.Filter - This class lets filter traces based on combinations of the filename, line number, and domain. It has the below-mentioned parameters.
domain
, filename_pattern
, and lineno
attributes.We have explained the usage of filters through the examples below. The first snapshot includes all traces for memory blocks used by python. The second snapshot includes traces of memory blocks that are not created by python. It can be due to C as numpy is built on it. The third snapshot includes entries where the filename is tracemalloc_ex5.py
and the fourth snapshot excludes entries with that file names.
CODE
import tracemalloc
import numpy as np
tracemalloc.start(10)
l2 = [i*i for i in range(10000)]
l1 = [i for i in range(10000)]
l3 = [i*i*i for i in range(10000)]
l4 = [i*2 for i in range(10000)]
l5 = np.random.randint(1,100, (1000,))
snapshot = tracemalloc.take_snapshot()
print("========== Original Snapshot ===========")
for stat in snapshot.statistics("lineno"):
print(stat)
print("\n========= Filtered Snapshot 1 =============")
filtr = tracemalloc.DomainFilter(inclusive=True, domain=0)
snap_shot = snapshot.filter_traces(filters=[filtr])
for stat in snap_shot.statistics("lineno"):
print(stat)
filtr = tracemalloc.DomainFilter(inclusive=False, domain=0)
snap_shot = snapshot.filter_traces(filters=[filtr])
print("\n========= Filtered Snapshot 2 =============")
for stat in snap_shot.statistics("lineno"):
print(stat)
filtr = tracemalloc.Filter(inclusive=True, filename_pattern="tracemalloc_ex5.py")
snap_shot = snapshot.filter_traces([filtr])
print("\n========= Filtered Snapshot 3 =============")
for stat in snap_shot.statistics("lineno"):
print(stat)
filtr = tracemalloc.Filter(inclusive=False, filename_pattern="tracemalloc_ex5.py")
snap_shot = snapshot.filter_traces([filtr])
print("\n========= Filtered Snapshot 4 =============")
for stat in snap_shot.statistics("lineno"):
print(stat)
OUTPUT
========== Original Snapshot ===========
tracemalloc_ex5.py:8: size=394 KiB, count=9994, average=40 B
tracemalloc_ex5.py:6: size=358 KiB, count=9984, average=37 B
tracemalloc_ex5.py:9: size=355 KiB, count=9872, average=37 B
tracemalloc_ex5.py:7: size=352 KiB, count=9744, average=37 B
tracemalloc_ex5.py:10: size=9050 B, count=5, average=1810 B
<__array_function__ internals>:6: size=536 B, count=1, average=536 B
/home/sunny/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2911: size=528 B, count=1, average=528 B
<__array_function__ internals>:4: size=464 B, count=1, average=464 B
/home/sunny/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:74: size=448 B, count=1, average=448 B
tracemalloc_ex5.py:12: size=432 B, count=1, average=432 B
/home/sunny/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:90: size=145 B, count=2, average=72 B
========= Filtered Snapshot 1 =============
tracemalloc_ex5.py:8: size=394 KiB, count=9994, average=40 B
tracemalloc_ex5.py:6: size=358 KiB, count=9984, average=37 B
tracemalloc_ex5.py:9: size=355 KiB, count=9872, average=37 B
tracemalloc_ex5.py:7: size=352 KiB, count=9744, average=37 B
tracemalloc_ex5.py:10: size=1050 B, count=4, average=262 B
<__array_function__ internals>:6: size=536 B, count=1, average=536 B
/home/sunny/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2911: size=528 B, count=1, average=528 B
<__array_function__ internals>:4: size=464 B, count=1, average=464 B
/home/sunny/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:74: size=448 B, count=1, average=448 B
tracemalloc_ex5.py:12: size=432 B, count=1, average=432 B
/home/sunny/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:90: size=145 B, count=2, average=72 B
========= Filtered Snapshot 2 =============
tracemalloc_ex5.py:10: size=8000 B, count=1, average=8000 B
========= Filtered Snapshot 3 =============
tracemalloc_ex5.py:8: size=394 KiB, count=9994, average=40 B
tracemalloc_ex5.py:6: size=358 KiB, count=9984, average=37 B
tracemalloc_ex5.py:9: size=355 KiB, count=9872, average=37 B
tracemalloc_ex5.py:7: size=352 KiB, count=9744, average=37 B
tracemalloc_ex5.py:10: size=9050 B, count=5, average=1810 B
tracemalloc_ex5.py:12: size=432 B, count=1, average=432 B
========= Filtered Snapshot 4 =============
<__array_function__ internals>:6: size=536 B, count=1, average=536 B
/home/sunny/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2911: size=528 B, count=1, average=528 B
<__array_function__ internals>:4: size=464 B, count=1, average=464 B
/home/sunny/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:74: size=448 B, count=1, average=448 B
/home/sunny/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:90: size=145 B, count=2, average=72 B
As a part of this example, we'll explain how we can compare two snapshots and find out the difference in traces between them.
key_type
parameter which will be used as the primary key to find out the difference between two snapshots. It'll return an object of class StatisticDiff
. This object has information about differences. Below we have explained through example, how we can get the difference between two snapshots. Please make a note that different traces taken before the first snapshot are not present.
CODE
import tracemalloc
tracemalloc.start()
l1 = [i for i in range(10000)]
l2 = [i*i for i in range(10000)]
l3 = [i*i*i for i in range(10000)]
snapshot1 = tracemalloc.take_snapshot()
print("SNAPSHOT - 1")
for stat in snapshot1.statistics("lineno"):
print(stat)
l4 = [i*i*i*i for i in range(10000)]
l5 = [i*i*i*i*i for i in range(10000)]
print("\nSNAPSHOT - 2")
snapshot2 = tracemalloc.take_snapshot()
for stat in snapshot2.statistics("lineno"):
#if "tracemalloc_ex6.py" in str(stat):
print(stat)
print("\nDIFFERENCE")
for diff in snapshot2.compare_to(snapshot1,"lineno"):
#if "tracemalloc_ex6.py" in str(diff):
print(diff)
OUTPUT
SNAPSHOT - 1
tracemalloc_ex6.py:7: size=394 KiB, count=9994, average=40 B
tracemalloc_ex6.py:6: size=358 KiB, count=9984, average=37 B
tracemalloc_ex6.py:5: size=352 KiB, count=9745, average=37 B
SNAPSHOT - 2
tracemalloc_ex6.py:17: size=436 KiB, count=9998, average=45 B
tracemalloc_ex6.py:16: size=433 KiB, count=9997, average=44 B
tracemalloc_ex6.py:7: size=394 KiB, count=9994, average=40 B
tracemalloc_ex6.py:6: size=358 KiB, count=9984, average=37 B
tracemalloc_ex6.py:5: size=352 KiB, count=9745, average=37 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:209: size=864 B, count=2, average=432 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:165: size=832 B, count=2, average=416 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:479: size=576 B, count=3, average=192 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:507: size=576 B, count=1, average=576 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:397: size=528 B, count=3, average=176 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:472: size=480 B, count=2, average=240 B
tracemalloc_ex6.py:12: size=464 B, count=1, average=464 B
tracemalloc_ex6.py:13: size=456 B, count=1, average=456 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:53: size=448 B, count=1, average=448 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:509: size=424 B, count=1, average=424 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:194: size=424 B, count=1, average=424 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:54: size=424 B, count=1, average=424 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:475: size=416 B, count=1, average=416 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:65: size=144 B, count=2, average=72 B
tracemalloc_ex6.py:11: size=106 B, count=2, average=53 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:508: size=64 B, count=1, average=64 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:534: size=56 B, count=1, average=56 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:185: size=56 B, count=1, average=56 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:291: size=40 B, count=1, average=40 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:477: size=28 B, count=1, average=28 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:476: size=28 B, count=1, average=28 B
DIFFERENCE
tracemalloc_ex6.py:17: size=436 KiB (+436 KiB), count=9998 (+9998), average=45 B
tracemalloc_ex6.py:16: size=433 KiB (+433 KiB), count=9997 (+9997), average=44 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:209: size=864 B (+864 B), count=2 (+2), average=432 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:165: size=832 B (+832 B), count=2 (+2), average=416 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:479: size=576 B (+576 B), count=3 (+3), average=192 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:507: size=576 B (+576 B), count=1 (+1), average=576 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:397: size=528 B (+528 B), count=3 (+3), average=176 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:472: size=480 B (+480 B), count=2 (+2), average=240 B
tracemalloc_ex6.py:12: size=464 B (+464 B), count=1 (+1), average=464 B
tracemalloc_ex6.py:13: size=456 B (+456 B), count=1 (+1), average=456 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:53: size=448 B (+448 B), count=1 (+1), average=448 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:509: size=424 B (+424 B), count=1 (+1), average=424 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:194: size=424 B (+424 B), count=1 (+1), average=424 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:54: size=424 B (+424 B), count=1 (+1), average=424 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:475: size=416 B (+416 B), count=1 (+1), average=416 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:65: size=144 B (+144 B), count=2 (+2), average=72 B
tracemalloc_ex6.py:11: size=106 B (+106 B), count=2 (+2), average=53 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:508: size=64 B (+64 B), count=1 (+1), average=64 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:534: size=56 B (+56 B), count=1 (+1), average=56 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:185: size=56 B (+56 B), count=1 (+1), average=56 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:291: size=40 B (+40 B), count=1 (+1), average=40 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:477: size=28 B (+28 B), count=1 (+1), average=28 B
/home/sunny/anaconda3/lib/python3.7/tracemalloc.py:476: size=28 B (+28 B), count=1 (+1), average=28 B
tracemalloc_ex6.py:7: size=394 KiB (+0 B), count=9994 (+0), average=40 B
tracemalloc_ex6.py:6: size=358 KiB (+0 B), count=9984 (+0), average=37 B
tracemalloc_ex6.py:5: size=352 KiB (+0 B), count=9745 (+0), average=37 B
Our seventh example is exactly the same as our sixth example with the only difference that we have used a filter to remove entry related to the tracemalloc module itself.
We have specified tracemalloc filename (tracemalloc.__file__
) to filename_pattern
of Filter class with the inclusive attribute as False to exclude entries with tracemalloc module.
CODE
import tracemalloc
tracemalloc.start()
l1 = [i for i in range(10000)]
l2 = [i*i for i in range(10000)]
l3 = [i*i*i for i in range(10000)]
snapshot1 = tracemalloc.take_snapshot()
print("SNAPSHOT - 1")
for stat in snapshot1.statistics("lineno"):
print(stat)
l4 = [i*i*i*i for i in range(10000)]
l5 = [i*i*i*i*i for i in range(10000)]
print("\nSNAPSHOT - 2")
snapshot2 = tracemalloc.take_snapshot()
filtr = tracemalloc.Filter(inclusive=False, filename_pattern=tracemalloc.__file__)
snapshot2 = snapshot2.filter_traces([filtr])
for stat in snapshot2.statistics("lineno"):
print(stat)
print("\nDIFFERENCE")
for diff in snapshot2.compare_to(snapshot1,"lineno"):
print(diff)
OUTPUT
SNAPSHOT - 1
tracemalloc_ex7.py:7: size=394 KiB, count=9994, average=40 B
tracemalloc_ex7.py:6: size=358 KiB, count=9984, average=37 B
tracemalloc_ex7.py:5: size=352 KiB, count=9745, average=37 B
SNAPSHOT - 2
tracemalloc_ex7.py:17: size=436 KiB, count=9998, average=45 B
tracemalloc_ex7.py:16: size=433 KiB, count=9997, average=44 B
tracemalloc_ex7.py:7: size=394 KiB, count=9994, average=40 B
tracemalloc_ex7.py:6: size=358 KiB, count=9984, average=37 B
tracemalloc_ex7.py:5: size=352 KiB, count=9745, average=37 B
tracemalloc_ex7.py:12: size=464 B, count=1, average=464 B
tracemalloc_ex7.py:13: size=456 B, count=1, average=456 B
tracemalloc_ex7.py:11: size=106 B, count=2, average=53 B
DIFFERENCE
tracemalloc_ex7.py:17: size=436 KiB (+436 KiB), count=9998 (+9998), average=45 B
tracemalloc_ex7.py:16: size=433 KiB (+433 KiB), count=9997 (+9997), average=44 B
tracemalloc_ex7.py:12: size=464 B (+464 B), count=1 (+1), average=464 B
tracemalloc_ex7.py:13: size=456 B (+456 B), count=1 (+1), average=456 B
tracemalloc_ex7.py:11: size=106 B (+106 B), count=2 (+2), average=53 B
tracemalloc_ex7.py:7: size=394 KiB (+0 B), count=9994 (+0), average=40 B
tracemalloc_ex7.py:6: size=358 KiB (+0 B), count=9984 (+0), average=37 B
tracemalloc_ex7.py:5: size=352 KiB (+0 B), count=9745 (+0), average=37 B
Our eighth example is also exactly the same as the seventh example but we have specified in a different way how we can include only entries pertaining to the file which we are tracing. We have specified tracemalloc_ex8.py
as a filename pattern to the Filter object. This way it'll eliminate entries of tracemalloc module and only keep the entry of this file.
CODE
import tracemalloc
tracemalloc.start()
l1 = [i for i in range(10000)]
l2 = [i*i for i in range(10000)]
l3 = [i*i*i for i in range(10000)]
snapshot1 = tracemalloc.take_snapshot()
print("SNAPSHOT - 1")
for stat in snapshot1.statistics("lineno"):
print(stat)
l4 = [i*i*i*i for i in range(10000)]
l5 = [i*i*i*i*i for i in range(10000)]
print("\nSNAPSHOT - 2")
snapshot2 = tracemalloc.take_snapshot()
filtr = tracemalloc.Filter(inclusive=True, filename_pattern="tracemalloc_ex8.py")
snapshot2 = snapshot2.filter_traces([filtr])
for stat in snapshot2.statistics("lineno"):
print(stat)
print("\nDIFFERENCE")
for diff in snapshot2.compare_to(snapshot1,"lineno"):
print(diff)
OUTPUT
SNAPSHOT - 1
tracemalloc_ex8.py:7: size=394 KiB, count=9994, average=40 B
tracemalloc_ex8.py:6: size=358 KiB, count=9984, average=37 B
tracemalloc_ex8.py:5: size=352 KiB, count=9745, average=37 B
SNAPSHOT - 2
tracemalloc_ex8.py:17: size=436 KiB, count=9998, average=45 B
tracemalloc_ex8.py:16: size=433 KiB, count=9997, average=44 B
tracemalloc_ex8.py:7: size=394 KiB, count=9994, average=40 B
tracemalloc_ex8.py:6: size=358 KiB, count=9984, average=37 B
tracemalloc_ex8.py:5: size=352 KiB, count=9745, average=37 B
tracemalloc_ex8.py:12: size=464 B, count=1, average=464 B
tracemalloc_ex8.py:13: size=456 B, count=1, average=456 B
tracemalloc_ex8.py:11: size=106 B, count=2, average=53 B
DIFFERENCE
tracemalloc_ex8.py:17: size=436 KiB (+436 KiB), count=9998 (+9998), average=45 B
tracemalloc_ex8.py:16: size=433 KiB (+433 KiB), count=9997 (+9997), average=44 B
tracemalloc_ex8.py:12: size=464 B (+464 B), count=1 (+1), average=464 B
tracemalloc_ex8.py:13: size=456 B (+456 B), count=1 (+1), average=456 B
tracemalloc_ex8.py:11: size=106 B (+106 B), count=2 (+2), average=53 B
tracemalloc_ex8.py:7: size=394 KiB (+0 B), count=9994 (+0), average=40 B
tracemalloc_ex8.py:6: size=358 KiB (+0 B), count=9984 (+0), average=37 B
tracemalloc_ex8.py:5: size=352 KiB (+0 B), count=9745 (+0), average=37 B
This ends our small tutorial explaining how to use tracemalloc in a different way to monitor memory usage in python. Please feel free to let us know your views in the comments section.