Basics of working with files in Python

Introduction

In this article you will learn how to organize working with files in Python 3.

Create the file files.py and copy the code from the examples there.

You can run the file with the command

python3 files.py

Create

You can create a file with the command open

Options:

r  reading

rb reading in binary mode

rt reading in text mode

w  write only.

wb writing in binary mode

wt writing in text mode

w+ writing and reading

a  writing to the end of the file - will preserve the data that was in the file

b - this is the binary mode selector

t - this is the text mode selector

Any option with w will overwrite the existing file - be careful!

Example:

f = open("log.txt","w+")

Encoding

docs.python.org/3/library/codecs.html#standard-encodings

import sys print(sys.getdefaultencoding())

utf-8

Open a file

Syntax:

open(path_to_file, mode, encoding)

By default the utf-8 encoding is used

To open a file for reading run

f = open("log.txt","r")

If the file log.txt does not exist, it will not be created

raceback (most recent call last): File "files.py", line 1, in <module> f = open('log.txt', 'r') FileNotFoundError: [Errno 2] No such file or directory: 'log.txt'

Close a file

To close the file run

f.close()

Context manager

It is preferable to use a context manager for working with files

As soon as you leave the block in which the file is open - it will be closed automatically

with open('text.txt', 'r') as f: pass print(f.closed)

True

With the function type() it is easy to make sure that the object obtained as a result of using with has the type _io.TextIOWrapper

For a complete picture, let's study the available methods and attributes using dir()

with open('text.txt', 'r') as f: print(type(f)) print(dir(f))

<class '_io.TextIOWrapper'> ['_CHUNK_SIZE', '__class__', '__del__', '__delattr__', '__dict__', '__dir__', '__doc__', '__enter__', '__eq__', '__exit__', '__format__', '__ge__', '__getattribute__', '__getstate__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__iter__', '__le__', '__lt__', '__ne__', '__new__', '__next__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '_checkClosed', '_checkReadable', '_checkSeekable', '_checkWritable', '_finalizing', 'buffer', 'close', 'closed', 'detach', 'encoding', 'errors', 'fileno', 'flush', 'isatty', 'line_buffering', 'mode', 'name', 'newlines', 'read', 'readable', 'readline', 'readlines', 'reconfigure', 'seek', 'seekable', 'tell', 'truncate', 'writable', 'write', 'write_through', 'writelines']

Since the object has the method next() it is convenient to iterate over it.

with open('text.txt', 'r') as f: print(next(f)) print(next(f)) print(next(f)) print(next(f))

topbicycle.ru heihei.ru eth1.ru Traceback (most recent call last): File "/mnt/c/Users/Andrei/open_ex.py", line 5, in print(next(f)) ^^^^^^^ StopIteration

next() works until the elements run out and then raises StopIteration an exception.

We will consider examples of working with files both with and without the context manager.

read(): reading a file

You can get data from a file using read()

You can pass an argument of type int to it and then the corresponding number of bytes of the file content will be returned.

Read the contents of the file

with open('sites.md', 'r') as f: f_contents = f.read() print(f_contents)

www.eth1.ru www.heihei.ru www.topbicycle.ru

Let's look at the file sites.md which consists of one line

topbicycle.ruheihei.rueth1.ru

>>> f = open('sites.md', mode='rt', encoding='utf-8') >>> f.read(3)

'top'

>>> f.read(7)

'bicycle'

>>> f.read(3)

'.ru'

To get everything that is left in the file you need to call read() without arguments.

>>> f.read()

'heihei.rueth1.ru'

If you call read() again, an empty string will be returned

>>> f.read()

''

At the moment the pointer is at the end of the file, but it can be moved using seek()

Read a certain number of characters

with open('sites.md', 'r') as f: f_contents = f.read(20) print(f_contents)

www.eth1.ru www.heihe

If you run this command sequentially - the following characters will be read

with open('sites.md', 'r') as f: f_contents = f.read(20) print(f_contents, end = '') f_contents = f.read(20) print(f_contents, end = '')

www.eth1.ru www.heihei.ru www.topbicycle.

Loop for an arbitrary number of characters .read

with open('sites.md', 'r') as f: size_to_read = 10 f_contents = f.read(size_to_read) while len(f_contents) > 0: print(f_contents, end = '') f_contents = f.read(size_to_read)

www.eth1.ru www.heihei.ru www.topbicycle.ru

The expression f_contents = f.read(size_to_read) is needed so that when the file ends and f.read(size_to_read) becomes zero len(f_contents) also becomes zero and the loop finishes

Empty a file

>>> f = open('sites.md', mode='w', encoding='utf-8') >>> f.write('')

0

>>> f.close() >>> exit()

readline(): reading line by line

The method readline() prints the content line by line.

And the method readlines() prints all lines as a list

Let's look at the file sites.md which consists of three lines

topbicycle.ru heihei.ru eth1.ru

>>> f = open('sites.md', mode='rt', encoding='utf-8') >>> f.readline()

'www.topbicycle.ru\n'

>>> f.readline()

'www.heihei.ru\n'

>>> f.readline()

'eth1.ru'

When the lines run out readline() will return not an error, but simply an empty string

>>> f.readline()

''

>>> type(f.readline())

<class 'str'>

If you need to read from the beginning again the method will come in handy seek() about which you can read in the article «Additional techniques for working with files»

>>> f.seek(0)

0

>>> f.readline()

'www.topbicycle.ru\n'

>>> f.close()

Lines one by one using readline() in the script

with open('sites.md', 'r') as f: f_contents = f.readline() print(f_contents) f_contents = f.readline() print(f_contents)

www.eth1.ru www.heihei.ru

You can remove the newline transitions using end=''

with open('sites.md', 'r') as f: f_contents = f.readline() print(f_contents, end = '') f_contents = f.readline() print(f_contents, end = '')

www.eth1.ru www.heihei.ru

Loop for printing line by line without using readline()

with open('sites.md', 'r') as f: for line in f: print(line, end = '')

www.eth1.ru www.heihei.ru www.topbicycle.ru

As an alternative you can use sys.stdout.write()

You can read about passing arguments from the terminal in the article sys.argv[]

# files.py import sys f = open(sys.argv[1], mode='rt', encoding='utf-8') for line in f: sys.stdout.write(line) f.close()

python files.py sites.md

www.eth1.ru www.heihei.ru www.topbicycle.ru

readlines()

The method readlines() prints all lines as a list

>>> f.readlines()

['www.topbicycle.ru\n', 'www.heihei.ru\n', 'eth1.ru']

>>> f.close()

All lines with the newline character - readlines

with open('sites.md', 'r') as f: f_contents = f.readlines() print(f_contents)

['www.eth1.ru\n', 'www.heihei.ru\n', 'www.topbicycle.ru\n']

File name

An example of a program that prints the file name and the mode in which it is opened

f = open('log.txt', 'r') print(f.name) print(f.mode) f.close()

If the file log.txt exists, then in the terminal you will see

log.txt
r

Copying files

Text files

They can be copied line by line

with open('sites.md', 'r') as rf: with open('sites_copy.txt', 'w') as wf: for line in rf: wf.write(line)

cat sites_copy.txt

www.eth1.ru www.heihei.ru www.topbicycle.ru

Images

They can also be copied line by line, but they must be opened and written in bit mode. That is, you need to add the option b

Download an image of a bicycle from the site TopBicycle.ru or take any other picture

wget https://topbicycle.ru/b/img/stels_pilot_950_MD_26.jpg
ls

stels_pilot_950_MD_26.jpg

with open('stels_pilot_950_MD_26.jpg', 'rb') as rf: with open('stels_pilot_950_MD_26_copy.jpg', 'wb') as wf: for line in rf: wf.write(line)

python3 files.py
ls

stels_pilot_950_MD_26.jpg stels_pilot_950_MD_26_copy.jpg

A more correct approach is considered to be copying not in line-by-line mode but in parts with a fixed size

with open('stels_pilot_950_MD_26.jpg', 'rb') as rf: with open('stels_pilot_950_MD_26_copy.jpg', 'wb') as wf: chunk_size = 4096 rf_chunk = rf.read(chunk_size) while len(rf_chunk) > 0: wf.write(rf_chunk) rf_chunk = rf.read(chunk_size)

Write a file

To clear the file of the old content and write new content into it the option is used w (write)

with open('log.txt', 'w') as f: f.write("some text")

Let's look at writing to a file in interactive mode and without the context manager

>>> f = open('sites.md', 'w') >>> f.write("topbicycle.ru")

13

>>> f.write("heihei.ru")

9

13 and 9 is the number of bytes passed to the file

>>> f.close() >>> exit() cat sites.md

cat sites.md www.topbicycle.ru www.heihei.ru eth1.ruwww.aviasales.ru www.booking.comwww.tutu.ru www.velodrive.ruwww.velosklad.ru%

topbicycle.ruheihei.ru

ls -l

-rw-r--r-- 1 andrei users 26 Nov 17 21:15 sites.md

The final size of the file will depend on the operating system.

In Windows and Linux there are different line breaks, therefore when Python applies its universal newline the number of bytes may increase by 1 or may stay the same.

write() returns the number of bytes passed to the file, not the actual size of the written data.

Append to a file

If you need to add new data to the previous content without deleting it - the option is used a (append)

with open('log.txt', 'a') as f: f.write("some text")

Let's look at an example of adding data to the file sites.md using the method writelines()

www.topbicycle.ru www.heihei.ru www.eth1.ru

>>> f = open('sites.md', mode='at', encoding='utf-8') >>> f.writelines(['www.aviasales.ru\n', 'www.booking.com', 'www.tutu.ru\n', 'www.velodrive.ru', 'www.velosklad.ru']) >>> f.close() >>> exit()

cat sites.md

www.topbicycle.ru www.heihei.ru eth1.ruwww.aviasales.ru www.booking.comwww.tutu.ru www.velodrive.ruwww.velosklad.ru

You can see that line breaks appeared only where you specified them manually.

If you need to add data not to the end of the file, but to a certain place - study the example in the article «Advanced techniques for working with files»

Article author: Andrei Olegovich