Share @ LinkedIn Facebook  shutil

Overview

shutil library helps the developer perform high-level operations like copying and removing on a bunch of files.

Note: High-level copying operations from a library does not copy metadata like ACLs, owner, group, etc.

In [1]:
import shutil
import glob
import os
data = """The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!"""
In [2]:
print('Creating directory structures and files for experimentation purpose.')
%mkdir folder_l1_1
%mkdir folder_l1_2
%mkdir folder_l1_1/folder_l2
!touch temp.txt temp.jpg temp.png a.png b.jpg c.txt d.txt t1.png t2.jpg .temp.png .temp.jpg
!touch folder_l1_1/t.txt folder_l1_1/t2.png folder_l1_1/a.mp4 folder_l1_1/b.mp4 folder_l1_1/c.mpeg folder_l1_1/t3.jpg
!touch folder_l1_2/t2.txt folder_l1_2/t1.png folder_l1_2/b.mp4 folder_l1_2/c.mp4 folder_l1_2/d.mpeg
!touch folder_l1_1/folder_l2/t.txt folder_l1_1/folder_l2/t2.png folder_l1_1/folder_l2/a.mp4
print('\nCurrent directory contents : ')
%ls
print('\nfolder_l1_1 directory contents : ')
%ls folder_l1_1
print('\nfolder_l1_2 directory contents :')
%ls folder_l1_2
print('\nfolder_l1_1/folder_l2 directory contents : ')
%ls folder_l1_1/folder_l2
Creating directory structures and files for experimentation purpose.

Current directory contents :
__notebook__.ipynb  a.png  c.txt  folder_l1_1/  t1.png  temp.jpg  temp.txt
__output__.json     b.jpg  d.txt  folder_l1_2/  t2.jpg  temp.png

folder_l1_1 directory contents :
a.mp4  b.mp4  c.mpeg  folder_l2/  t.txt  t2.png  t3.jpg

folder_l1_2 directory contents :
b.mp4  c.mp4  d.mpeg  t1.png  t2.txt

folder_l1_1/folder_l2 directory contents :
a.mp4  t.txt  t2.png
In [3]:
## Writing data string defined above to c.txt file.
with open('c.txt','w') as f:
    f.write(data)
In [4]:
!cat c.txt | wc -l ## Checking whether data got written to c.txt
20
  • shutil.copyfileobj(fsrc,fdest[,length]) - It helps copy data from source file object fsrc to destination file object fdest. If length is given then it copies data in chunks specified by length. If a negative value is given for length then it might try to copy whole data which might cause memory issues in case of big files. Also, copying starts from whichever position the file pointer is in the file. If it isn't at 0 positions then a copy will not start from the beginning. Default value of length is 16384.
In [5]:
fsrc = open('c.txt', 'r')
fdest = open('d.txt','a')
shutil.copyfileobj(fsrc,fdest)
In [6]:
!cat d.txt | wc -l
0
  • shutil.copyfile(src,dest,follow_symlinks=True) - Copies conttents of file named src to file dest. If follow_symlinks is set to False and src is symlinks then it creates new symlinks rather than copying contents. If src and dest both are same then it raises SameFileError.
In [7]:
shutil.copyfile('c.txt', 'e.txt') ## If destination file already exists then it overwrites and if it does not then creates one.
Out[7]:
'e.txt'
In [8]:
!cat e.txt | wc -l
20
  • shutil.copystat(src,dest,follow_symlinks=True) - Copies permission bits from src to dest. If src and dest both are symlinks and follow_symlinks is False then it tries to modify dest symlink not the original file it's pointing to.
  • shutil.copymode(src,dest,follow_symlinks=True) - Copies permission bits,last access time, last modified time and flag from src to dst

Both above methods do not modify file contents, owner and group.

In [9]:
## Please make a note that a.png and b.jpg has different permission bits. 
## a.png and d.jpg has difference permission bits, creation time as well
!chown sys a.png
!touch d.png b.png d.jpg
!chmod 655 a.png
!ls -lrt a.png d.jpg b.jpg
-rw-r--r-- 1 root root 0 Feb 18 03:04 b.jpg
-rw-r-xr-x 1 sys  root 0 Feb 18 03:04 a.png
-rw-r--r-- 1 root root 0 Feb 18 03:04 d.jpg
In [10]:
shutil.copystat('a.png','b.jpg')
shutil.copymode('a.png','d.jpg')
In [11]:
!ls -lrt a.png d.jpg b.jpg ## Please make a note in permission change of d.jpg and b.jpg at begining.
-rw-r-xr-x 1 root root 0 Feb 18 03:04 b.jpg
-rw-r-xr-x 1 sys  root 0 Feb 18 03:04 a.png
-rw-r-xr-x 1 root root 0 Feb 18 03:04 d.jpg
  • shutil.copy(src,dest,follow_symlinks=True) - Copies src to destination file or directory dest. If dest is directory then it'll copy src to that directory. If follow_symlinks is True then it copies original file from src to dest else it creates symlink.
  • shutil.copy2(src,dest,follow_symlinks=True) - Same as above method but tries to preserve metadata of src file into dest.It uses copystat() to copy metadata.
In [12]:
shutil.copy('d.jpg','e.jpg'), shutil.copy2('d.jpg','f.jpg')
Out[12]:
('e.jpg', 'f.jpg')
In [13]:
!ls -lrt d.jpg e.jpg f.jpg ## Please make a note that copy() did not copy metadata like creation/modifcation time. But copy2() did copy metadata.
-rw-r-xr-x 1 root root 0 Feb 18 03:04 f.jpg
-rw-r-xr-x 1 root root 0 Feb 18 03:04 d.jpg
-rw-r-xr-x 1 root root 0 Feb 18 03:04 e.jpg
  • shutil.disk_usage(path) - Returns disk usage stats of path as namedtuple. It returns status in bytes.
  • shutil.chown(path,user=None,group=None) - Change owner user and/or group of path.user and group can be system user name or uid or gid as well.
In [14]:
shutil.disk_usage(os.getcwd())
Out[14]:
usage(total=5217320960, used=10788864, free=4938096640)
In [15]:
!ls -lrt d.jpg e.jpg
shutil.chown('d.jpg',user='sys')
shutil.chown('e.jpg',user='sys', group='sys')
!ls -lrt d.jpg e.jpg
-rw-r-xr-x 1 root root 0 Feb 18 03:04 d.jpg
-rw-r-xr-x 1 root root 0 Feb 18 03:04 e.jpg
-rw-r-xr-x 1 sys root 0 Feb 18 03:04 d.jpg
-rw-r-xr-x 1 sys sys  0 Feb 18 03:04 e.jpg
  • shutil.which(cmd,mode=os.F_OK|os.X_OK,path=None) - It returns path of executable which will run if cmd gets executed in shell/command prompt. Developer can also provide path to look for other wise it'll use os.environ() path which will be PATH variable of system.
In [16]:
print(shutil.which('rm'))
print(shutil.which('ls'))
print(shutil.which('pip'))
print(shutil.which('conda'))
print(shutil.which('wheel'))
print(shutil.which('chown'))
print(shutil.which('chmod'))
print(shutil.which('jupyter'))
print(shutil.which('ipython'))
/bin/rm
/bin/ls
/opt/conda/bin/pip
/opt/conda/bin/conda
/opt/conda/bin/wheel
/bin/chown
/bin/chmod
/opt/conda/bin/jupyter
/opt/conda/bin/ipython

Sunny Solanki  Sunny Solanki