Testing a module loaded from a path and a shell helper

Introduction

Downloaded server logs do not belong in a repository: they are large, they change every day, and some of them hold client IP addresses. The prod-log pipeline keeps them in one external directory, and every script must agree on where that directory is.

The folder container/tools/analyze_logs answers the question twice: in Python, in log_paths.py, and in shell, in log-root.sh. The shell scripts source the second file, the Python tools import the first one.

Its test, container/tests/test_log_paths.py, is three tests long and shows a pattern that most test suites need sooner or later: one parametrized assertion, a module loaded from a file path, and a subprocess call to a shell helper.

Tools used here: Python , importlib , subprocess , pytest .

A parametrized assert

The resolver has four inputs that matter: the home directory, the value of XDG_DATA_HOME, the override AREDEL_LOG_DIR, and nothing else. A pure function with those inputs can be checked without a fixture at all. The parametrize marker turns one test into five, one per row, and pytest prints the row in the test id.

In the file test_log_root.py:

import os import pytest def log_root(home, xdg_data="", override=""): if override: path = home + override[1:] if override.startswith("~/") else override return os.path.normpath(path) base = xdg_data or os.path.join(home, ".local", "share") return os.path.join(base, "aredel", "logs") @pytest.mark.parametrize( ("home", "xdg_data", "override", "expected"), [ ("/home/user", "", "", "/home/user/.local/share/aredel/logs"), ("/home/user", "/data", "", "/data/aredel/logs"), ("/home/user", "", "~/custom-logs", "/home/user/custom-logs"), ("/home/user", "/data", "~/logs", "/home/user/logs"), ("/home/user", "", "/var/log/aredel/", "/var/log/aredel"), ], ) def test_log_root(home, xdg_data, override, expected): assert log_root(home, xdg_data, override) == expected

python -m pytest test_log_root.py -v

============================= test session starts ============================== platform linux -- Python 3.12.16, pytest-9.2.0, pluggy-1.7.0 -- … rootdir: <project> plugins: allure-pytest-2.16.2 collecting ... collected 5 items test_log_root.py::test_log_root[/home/user---/home/user/.local/share/aredel/logs] PASSED [ 20%] test_log_root.py::test_log_root[/home/user-/data--/data/aredel/logs] PASSED [ 40%] test_log_root.py::test_log_root[/home/user--~/custom-logs-/home/user/custom-logs] PASSED [ 60%] test_log_root.py::test_log_root[/home/user-/data-~/logs-/home/user/logs] PASSED [ 80%] test_log_root.py::test_log_root[/home/user--/var/log/aredel/-/var/log/aredel] PASSED [100%] ============================== 5 passed in 0.05s ===============================

The ids are ugly because the values are paths, but they are honest: a failure names the exact row. Two rows carry the behaviour that is easy to get wrong: a trailing slash is removed (/var/log/aredel/ becomes /var/log/aredel), and a leading ~/ is expanded against the given home, not against the home of the machine that runs the test.

The catch of this style is that it tests a copy of the logic. The function above lives in the test file, so the real module can drift away from it without a red test. That is why the rest of the file tests the real module, loaded from its real path.

Load the module from a path

log_paths.py is not an installed package and its folder is not on sys.path. A plain import log_paths would fail; the test loads the file by path instead. The importlib recipe is three calls: build a spec, build a module object from it, run the file in that object.

The module container/tools/analyze_logs/log_paths.py (abridged):

import os def log_root() -> str: override = os.environ.get("AREDEL_LOG_DIR", "").strip() if override: return os.path.abspath(os.path.expanduser(override)) xdg_data = os.environ.get("XDG_DATA_HOME", "").strip() base = os.path.expanduser(xdg_data) if xdg_data else os.path.join(os.path.expanduser("~"), ".local", "share") return os.path.abspath(os.path.join(base, "aredel", "logs")) def prod_dir() -> str: return os.path.join(log_root(), "prod") def prod_access_dir() -> str: return os.path.join(prod_dir(), "access")

The test file test_log_paths.py (first half):

import importlib.util from pathlib import Path ANALYZE_LOGS = Path(__file__).resolve().parent / "analyze_logs" def load_log_paths(): spec = importlib.util.spec_from_file_location("aredel_log_paths", ANALYZE_LOGS / "log_paths.py") assert spec is not None and spec.loader is not None module = importlib.util.module_from_spec(spec) spec.loader.exec_module(module) return module def test_default_log_root_uses_xdg_data_home(monkeypatch, tmp_path): monkeypatch.delenv("AREDEL_LOG_DIR", raising=False) monkeypatch.setenv("XDG_DATA_HOME", str(tmp_path / "xdg")) monkeypatch.setenv("HOME", str(tmp_path / "home")) log_paths = load_log_paths() assert log_paths.log_root() == str(tmp_path / "xdg" / "aredel" / "logs") assert log_paths.prod_access_dir() == str(tmp_path / "xdg" / "aredel" / "logs" / "prod" / "access") def test_override_expands_home(monkeypatch, tmp_path): monkeypatch.setenv("HOME", str(tmp_path / "home")) monkeypatch.setenv("AREDEL_LOG_DIR", "~/custom-logs") log_paths = load_log_paths() assert log_paths.log_root() == str(tmp_path / "home" / "custom-logs")

python -m pytest test_log_paths.py -v

Three details make the loader safe:

Path(__file__).resolve() gives the folder of the test file, so the path to the module does not depend on the working directory pytest was started from;
the loader is a function, not a module-level constant: each test that calls it gets a fresh module object, and a value cached in the module cannot leak into the next test;
monkeypatch.delenv("AREDEL_LOG_DIR", raising=False) is needed even though the test sets the two variables it uses. On a machine where the developer exported AREDEL_LOG_DIR for real, the override would win and the default branch would never run.

Call the shell helper

The Python module and the shell file must agree on the same formula. A test of the Python side alone would not notice a change in log-root.sh, and the shell scripts would then write the logs to a different directory than the Python reports read. The test sources the shell file in a fresh bash and asks it for the variable:

import os import subprocess from pathlib import Path ANALYZE_LOGS = Path(__file__).resolve().parent / "analyze_logs" LOG_ROOT_SH = ANALYZE_LOGS / "log-root.sh" def shell_log_root(env): result = subprocess.run( ["bash", "-c", f'source "{LOG_ROOT_SH}"; printf "%s" "$AREDEL_LOG_DIR"'], capture_output=True, check=True, env=env, text=True, ) return result.stdout def test_shell_helper_matches_python_default(tmp_path): env = {**os.environ, "XDG_DATA_HOME": str(tmp_path / "xdg"), "HOME": str(tmp_path / "home")} assert shell_log_root(env) == str(tmp_path / "xdg" / "aredel" / "logs") def test_shell_helper_expands_tilde(tmp_path): env = {**os.environ, "HOME": str(tmp_path / "home"), "AREDEL_LOG_DIR": "~/custom-logs"} assert shell_log_root(env) == str(tmp_path / "home" / "custom-logs")

python -m pytest test_log_root_sh.py -v

The arguments of subprocess.run carry the meaning:

capture_output=True keeps the output in result.stdout instead of printing it into the test log;
text=True decodes the output to str; without it result.stdout is bytes and the comparison fails;
check=True turns a non-zero exit code into CalledProcessError, so a broken script fails the test with its own error instead of comparing an empty string;
env=env replaces the whole environment of the child, which is why the example starts from {**os.environ, ...}: the child still needs PATH to find bash.

printf is used instead of echo because the value is a path: echo adds a newline, and the assertion would have to strip it.

The real test in the repo

The real file combines the three ideas and adds a few lines that only the repository needs. The path is built as Path(__file__).resolve().parents[1] / "tools" / "analyze_logs", so it points outside the tests folder, and the shell test also compares against the Python module instead of a fixed string, which pins the two implementations to each other.

The test file container/tests/test_log_paths.py:

from __future__ import annotations import importlib.util import os import subprocess from pathlib import Path ANALYZE_LOGS = Path(__file__).resolve().parents[1] / "tools" / "analyze_logs" LOG_ROOT_SH = ANALYZE_LOGS / "log-root.sh" def _load_log_paths(): spec = importlib.util.spec_from_file_location("aredel_log_paths", ANALYZE_LOGS / "log_paths.py") assert spec is not None and spec.loader is not None module = importlib.util.module_from_spec(spec) spec.loader.exec_module(module) return module def test_default_log_root_uses_xdg_data_home(monkeypatch, tmp_path): monkeypatch.delenv("AREDEL_LOG_DIR", raising=False) monkeypatch.setenv("XDG_DATA_HOME", str(tmp_path / "xdg")) monkeypatch.setenv("HOME", str(tmp_path / "home")) log_paths = _load_log_paths() assert log_paths.log_root() == str(tmp_path / "xdg" / "aredel" / "logs") assert log_paths.prod_access_dir() == str(tmp_path / "xdg" / "aredel" / "logs" / "prod" / "access") assert log_paths.prod_error_dir() == str(tmp_path / "xdg" / "aredel" / "logs" / "prod" / "error") assert log_paths.devices_dir() == str(tmp_path / "xdg" / "aredel" / "logs" / "devices") assert log_paths.user_profiles_dir() == str(tmp_path / "xdg" / "aredel" / "logs" / "user_profiles") assert log_paths.botlist_path() == str(tmp_path / "xdg" / "aredel" / "logs" / "botlist.csv") def test_override_expands_home(monkeypatch, tmp_path): monkeypatch.setenv("HOME", str(tmp_path / "home")) monkeypatch.setenv("AREDEL_LOG_DIR", "~/custom-logs") log_paths = _load_log_paths() assert log_paths.log_root() == str(tmp_path / "home" / "custom-logs") def test_shell_helper_matches_python_default(monkeypatch, tmp_path): monkeypatch.delenv("AREDEL_LOG_DIR", raising=False) monkeypatch.setenv("XDG_DATA_HOME", str(tmp_path / "xdg")) monkeypatch.setenv("HOME", str(tmp_path / "home")) log_paths = _load_log_paths() env = {**os.environ, "XDG_DATA_HOME": str(tmp_path / "xdg"), "HOME": str(tmp_path / "home")} result = subprocess.run( ["bash", "-c", f'source "{LOG_ROOT_SH}"; printf "%s" "$AREDEL_LOG_DIR"'], capture_output=True, check=True, env=env, text=True, ) assert result.stdout == log_paths.log_root()

The differences from the examples above each have a reason:

the loader in the repo takes no argument and is called by every test, so a test can set the environment first and load the module second;
test_default_log_root_uses_xdg_data_home checks all five directories, not just the root: the helpers under it are one-line joins, and a typo there changes where a whole pipeline writes;
the shell test sets the same variables with monkeypatch and then builds the child environment by hand, because monkeypatch cannot change the environment of a process that has not started yet.

Run the file from the container directory:

./tools/run-tests-venv.sh -- tests/test_log_paths.py -v -p no:cacheprovider

This run was made with pytest 9.2.0 on Python 3.12.16 from a scratch venv with the repository container/tests/requirements.txt installed, which is why the run prints an installed allure-pytest plugin. The -p no:cacheprovider is there because the checkout used for this run was mounted read-only and pytest could not create its cache directory; on the host ./tools/run-tests-venv.sh -- tests/test_log_paths.py -v is enough. The three tests are the three layers: a pure function checked from a table, a module loaded from a file path, and a shell script that must agree with it.

Article author: Arthur Isaev