Python is a versatile programming language that not only excels in data analysis, web development, and scripting but also provides the ability to interact directly with the operating system. One of the powerful features of Python is its capability to execute shell or Linux commands directly from within a Python script. This functionality can be very useful for automating system tasks, managing files, or integrating Python with other tools. In this article, we’ll explore various methods for executing shell/Linux commands in Python.

Python offers several ways to execute shell commands. The most common methods include:

1) Using the os.system() Method
2) Using the os.popen() Method
3) Using the subprocess.run() Method
4) Using the subprocess.Popen() Method

Let’s dive into each method:

1. Using the os.system() method: This method is one of the simplest ways to execute a shell command in Python. It runs the command in a subshell and returns the exit status of the command.

import os

# Execute a simple shell command
os.system('ls -l')

# return status : 0 -> Successful execution of the command
# return status : 1 -> Failed execution of the command

Output:

0

In this example, the ls -l command is executed, and its output is printed to the terminal. While os.system() is easy to use, it has limitations, such as not capturing the command’s output directly within Python. For more advanced use cases, the subprocess module is preferred.

2. Using the os.popen() method: This method is a legacy method that opens a pipe to or from a command. It returns a file-like object that can be read or written.

import os

# Execute a shell command and read the output
with os.popen('ls -l') as stream:
    output = stream.read()

print(output)

Output:

total 12
-rw------- 1 runner runner 365 Dec 17  2023 main.py
-rw------- 1 runner runner 254 Dec 12  2023 poetry.lock
-rw------- 1 runner runner 544 Dec 12  2023 pyproject.toml

While os.popen() can be useful for simple tasks, it’s generally recommended to use the subprocess module for greater control and flexibility.

3. Using the subprocess.run() method: This is the recommended method for running shell/linux commands. It executes the command, waits for it to complete, and returns a CompletedProcess object, which contains information about the execution.

import subprocess

# Execute a shell command and capture the output
result = subprocess.run(['ls', '-l'], capture_output=True, text=True)

# Print the output
print(result.stdout)

Output:

total 12
-rw------- 1 runner runner 365 Dec 17  2023 main.py
-rw------- 1 runner runner 254 Dec 12  2023 poetry.lock
-rw------- 1 runner runner 544 Dec 12  2023 pyproject.toml

In this example, the ls -l command is executed, and its output is captured and printed. The capture_output=True parameter ensures that both stdout and stderr are captured, and text=True returns the output as a string.

4. Using the subprocess.Popen() method: For more advanced use cases, such as running commands in the background or piping data between commands, subprocess.Popen() offers greater control.

import subprocess

# Run a command and capture the output
process = subprocess.Popen(['ls', '-l'], stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)

# Get the output and error
stdout, stderr = process.communicate()

print("Output:\n", stdout)
print("Error:\n", stderr)

Output:

Output:
 total 12
-rw------- 1 runner runner 365 Dec 17  2023 main.py
-rw------- 1 runner runner 254 Dec 12  2023 poetry.lock
-rw------- 1 runner runner 544 Dec 12  2023 pyproject.toml
Error:

Here, subprocess.Popen() runs the ls -l command and captures both the output and error streams. The communicate() method is used to interact with the process and retrieve the output

If you need to run a command through the shell (e.g., to use shell features like wildcard expansion), you can set shell=True in subprocess.run() or subprocess.Popen().

import subprocess

# Execute a shell command with shell=True
subprocess.run('echo Hello, Shell!', shell=True)

Output:

Hello, Shell!
CompletedProcess(args='echo Hello, Shell!', returncode=0)

Conclusion: Executing shell or Linux commands directly from Python scripts opens up a world of possibilities for automation, system administration, and task integration. Whether you need to run a simple command or manage complex processes, Python provides the tools to do so efficiently. The subprocess module, in particular, is a powerful and flexible way to execute and manage shell commands in Python, offering fine-grained control over input, output, and error handling.