subprocess python

Introduction

In this article you will learn how to run Linux and Windows commands from Python 3 code.

Create the file subprocess_lesson.py and copy the code from the examples into it.

The file can be run with the command python3 subprocess.py

A simple example

An example of a program that runs the Linux command ls

import subprocess subprocess.run('ls')

A simple example for Windows

An example of a program that runs in Windows the command dir

import subprocess subprocess.run('dir', shell=True)

It does not work for me yet

A Bash command with options

To run a Bash command with options, for example, ls - la you need to add shell=True

import subprocess subprocess.run('ls -la', shell=True)

Using shell=True has one important feature: you need to watch security especially carefully.

It is recommended to use shell=True only if you pass the parameters yourself.

Using shell=True can be avoided by passing the command and parameters as a list:

import subprocess subprocess.run(['ls', '-la'])

Pass a variable as a command argument

By analogy with the previous paragraph, a variable can also be used as an argument

import subprocess text = "Visit TopBicycle.ru to support my website" subprocess.run(["echo",text])

python3 subprocess_lesson.py

Visit TopBicycle.ru to support my website

args, returncode, stdout

Let us examine subprocess in more detail

import subprocess p1 = subprocess.run(['ls', '-la']) print("p1") print(p1) print("p1.args") print(p1.args) print("p1.returncode") print(p1.returncode) print("p1.stdout") print(p1.stdout)

python3 subprocess_lesson.py

total 12 drwxrwxr-x 2 andrei andrei 4096 Nov 30 17:57 . drwxrwxr-x 3 andrei andrei 4096 Nov 30 17:57 .. -rw-rw-r-- 1 andrei andrei 195 Nov 30 16:51 subprocess_lesson.py p1 CompletedProcess(args='ls -la', returncode=0) p1.args ls -la p1.returncode 0 p1.stdout None

To not output the result to the terminal but save it in a variable, you need to use the option capture_output=True

import subprocess p1 = subprocess.run(['ls', '-la'], capture_output=True) print(p1.stdout)

python3 subprocess_lesson.py

b'total 12\ndrwxrwxr-x 2 andrei andrei 4096 Nov 30 18:41 .\ndrwxrwxr-x 3 andrei andrei 4096 Nov 30 18:40 ..\n-rw-rw-r-- 1 andrei andrei 92 Nov 30 18:41 subprocess_lesson.py\n'

If you do not like the byte output you can decode it

import subprocess p1 = subprocess.run(['ls', '-la'], capture_output=True) print(p1.stdout.decode())

python3 subprocess_lesson.py

total 12 drwxrwxr-x 2 andrei andrei 4096 Nov 30 18:41 . drwxrwxr-x 3 andrei andrei 4096 Nov 30 18:40 .. -rw-rw-r-- 1 andrei andrei 101 Nov 30 18:46 subprocess_lesson.py

Or you can use the option text=True

import subprocess p1 = subprocess.run(['ls', '-la'], capture_output=True, text=True) print(p1.stdout)

total 12 drwxrwxr-x 2 andrei andrei 4096 Nov 30 18:41 . drwxrwxr-x 3 andrei andrei 4096 Nov 30 18:40 .. -rw-rw-r-- 1 andrei andrei 101 Nov 30 18:46 subprocess_lesson.py

Another variant of redirecting the output is stdout=subprocess.PIPE

import subprocess p1 = subprocess.run(['ls', '-la'], stdout=subprocess.PIPE, text=True) print(p1.stdout)

python3 subprocess_lesson.py

total 12 drwxrwxr-x 2 andrei andrei 4096 Nov 30 18:41 . drwxrwxr-x 3 andrei andrei 4096 Nov 30 18:40 .. -rw-rw-r-- 1 andrei andrei 101 Nov 30 18:46 subprocess_lesson.py

Output to a file

import subprocess with open('output.txt', 'w') as f: p1 = subprocess.run(['ls', '-la'], stdout=f, text=True)

Error handling

Let us add a deliberately wrong condition to the command. For example, let the listing be run not for the current directory but for a non-existent one.

import subprocess p1 = subprocess.run(['ls', '-la', 'not_exist'], capture_output=True, text=True) print(p1.returncode) print(p1.stderr)

2 ls: cannot access 'not_exist': No such file or directory

Note that Python in this example does not produce any errors

To make Python report errors in external commands, use the option check=True

import subprocess p1 = subprocess.run(['ls', '-la', 'not_exist'], capture_output=True, text=True, check=True) print(p1.returncode) print(p1.stderr)

python3 subprocess_lesson.py

Traceback (most recent call last): File "subprocess_lesson.py", line 3, in <module> p1 = subprocess.run(['ls', '-la', 'not_exist'], capture_output=True, text=True, check=True) File "/usr/lib/python3.8/subprocess.py", line 512, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['ls', '-la', 'not_exist']' returned non-zero exit status 2.

Note that now Python produces an error, and it never reaches print(p1.returncode) and print(p1.stderr)

import subprocess p1 = subprocess.run(['ls', '-la', 'not_exist'], stderr=subprocess.DEVNULL) print(p1.stderr)

python3 subprocess_lesson.py

None

Passing arguments to a script

Suppose you need to call a script with several arguments

import subprocess subprocess.call(['./script.sh %s %s %s' %(ip, username, test_time)], shell=True)

Another example: call sed from a python script and cut the number of lines that the script itself receives as an argument

import subprocess LINES = int(sys.argv[1]) subprocess.call(['sed -i -e 1,%sd 2026-11-17-log.txt' %(LINES)], shell=True)

This problem can be solved in pure Python solution

with open('file_with_lines.txt', 'r') as fin: data = fin.readlines()[3:] with open('file_with_lines.txt', 'w') as fout: fout.writelines(data)

Logs with subprocess

If you run code in some environment where logging to a file is inconvenient and logging with print is impossible, you can use echo from bash

import subprocess text = "Andrei Log: robot/src/libraries/TestController.py is running" subprocess.run(["echo", text])

Compare two files

If you run code in some environment where logging to a file is inconvenient and logging with print is impossible, you can use echo from bash

import subprocess def compare(file1, file2): subprocess.run(["diff", file1, file2])

Article author: Andrei Olegovich