Testing a module that reads a setting at import

Introduction

The sitemap tests request every URL of the site. Sometimes you need to run them over one folder only, for example while a page is being rewritten. For that the suite has an optional page subset: the module container/tests/url_selection.py reads a file of paths from the environment and answers whether a URL is in it.

Its test, container/tests/test_url_selection.py, is the smallest test file of the suite: two tests, three imports, no web server, no browser, no PHP. It still shows three useful things: a plain assert, a temporary file, and a module that reads its setting once, at import time.

Tools used here: Python , pytest .

A test that is only an assert

A pytest test is a function whose name starts with test_. No class and no base class are needed, and the file is run by the test runner itself.

In the file test_math.py:

def test_sum(): assert sum([1, 2, 3]) == 6, "Should be 6" def test_sum_tuple(): assert sum((1, 2, 2)) == 6, "Should be 6"

Run it:

python -m pytest test_math.py -q

.F [100%] =================================== FAILURES =================================== ________________________________ test_sum_tuple ________________________________ def test_sum_tuple(): > assert sum((1, 2, 2)) == 6, "Should be 6" E AssertionError: Should be 6 E assert 5 == 6 E + where 5 = sum((1, 2, 2)) test_math.py:6: AssertionError =========================== short test summary info ============================ FAILED test_math.py::test_sum_tuple - AssertionError: Should be 6 1 failed, 1 passed in 0.11s

The message after the comma is what the reader sees in the failure: AssertionError: Should be 6. The next two lines are pytest explaining the values on its own: assert 5 == 6 and where the 5 came from. That is the whole reason the built-in assert needs no special methods.

One fixture: tmp_path

The real test needs a file with paths on disk. pytest gives every test its own empty directory through the tmp_path fixture: the test asks for the argument, and pytest creates and later removes the directory.

In the file test_listed_paths.py:

def test_paths_file_reads_only_paths(tmp_path): listed = tmp_path / "paths.txt" listed.write_text("/about/\n/ru/qa/page.php\n# comment\nnot-a-path\n", encoding="utf-8") paths = [ line.strip() for line in listed.read_text(encoding="utf-8").splitlines() if line.startswith("/") ] assert paths == ["/about/", "/ru/qa/page.php"]

python -m pytest test_listed_paths.py -q

. [100%] 1 passed in 0.03s

A fixture is used by naming it in the test signature. Nothing is imported and nothing is passed by hand: pytest sees the argument name and builds the value.

The setting is read at import

Now the part that makes the real test interesting. A module may read an environment variable while it is being imported, once, at module level.

In the file selection.py:

import os PATHS_FILE = os.environ.get("PATHS_FILE", "").strip() SELECTED = None if PATHS_FILE: with open(PATHS_FILE, encoding="utf-8") as handle: SELECTED = frozenset(line.strip() for line in handle if line.startswith("/")) def in_selection(path): return SELECTED is None or path in SELECTED

When PATHS_FILE is unset, the module accepts every path. When it names a file, only the listed paths pass. The obvious test sets the variable first and then asks the module:

import selection def test_env_change_after_import(monkeypatch, tmp_path): listed = tmp_path / "paths.txt" listed.write_text("/about/\n", encoding="utf-8") monkeypatch.setenv("PATHS_FILE", str(listed)) assert selection.in_selection("/about/") assert not selection.in_selection("/team/")

The monkeypatch fixture sets the variable and removes it again afterwards.

python -m pytest test_selection.py -q

F [100%] =================================== FAILURES =================================== _________________________ test_env_change_after_import _________________________ monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x73e5deee4ef0> tmp_path = PosixPath('<tmp>/test_env_change_after_import0') def test_env_change_after_import(monkeypatch, tmp_path): listed = tmp_path / "paths.txt" listed.write_text("/about/\n", encoding="utf-8") monkeypatch.setenv("PATHS_FILE", str(listed)) assert selection.in_selection("/about/") > assert not selection.in_selection("/team/") E AssertionError: assert not True E + where True = <function in_selection at 0x73e5deecf100>('/team/') E + where <function in_selection at 0x73e5deecf100> = selection.in_selection test_selection.py:10: AssertionError =========================== short test summary info ============================ FAILED test_selection.py::test_env_change_after_import - AssertionError: asse... 1 failed in 0.10s

The first assert passes and the second one fails. The module was imported before the test ran, at collection time, when PATHS_FILE was still unset. It stored SELECTED = None and now accepts /team/ as well. Changing os.environ afterwards cannot change a value that is already computed.

Note what the first assert alone would have proved: nothing. With SELECTED = None every path passes, so a test that only checks the listed path is green for the wrong reason. The negative assertion is the one that catches the stale module.

Reload the module

importlib.reload() runs the module body again in the same module object. The second run sees the environment the test has just set, so SELECTED is computed from the file.

import importlib import pytest import selection as selection_module def load_selection(monkeypatch, tmp_path): listed = tmp_path / "paths.txt" listed.write_text("/about/\n", encoding="utf-8") monkeypatch.setenv("PATHS_FILE", str(listed)) return importlib.reload(selection_module) def test_reload_picks_up_the_variable(monkeypatch, tmp_path): selection = load_selection(monkeypatch, tmp_path) assert selection.in_selection("/about/") assert not selection.in_selection("/team/") def test_unset_variable_selects_everything(monkeypatch): monkeypatch.delenv("PATHS_FILE", raising=False) selection = importlib.reload(selection_module) assert selection.in_selection("/anything/")

python -m pytest test_selection.py -q

.. [100%] 2 passed in 0.04s

The module is imported as selection as selection_module on purpose: each test receives the reloaded module from the helper, and the name selection stays free for the local variable, so a reader cannot mistake one for the other.

Put the module back

A reload changes the module for the whole pytest session. Without cleanup, a later test would see the paths file of this one. An autouse fixture reloads the module once more after the test, when the environment is back to its original state:

@pytest.fixture(autouse=True) def restore_module(): yield importlib.reload(selection_module) # runs after monkeypatch restored the env

autouse=True applies the fixture to every test in the file without naming it. The code after yield runs at teardown. pytest tears fixtures down in the reverse order of setup, and an autouse fixture is set up before the monkeypatch a test asks for, so it is torn down after monkeypatch has already put the variable back. The reload therefore reads the original environment, not the test one.

The real test in the repo

The repo module does a little more than the example: it decodes the URL, treats /dir/index.php and /dir/ as one page, ignores lines that do not start with a slash, and skips a test instead of failing it when the subset has nothing to check.

The module container/tests/url_selection.py:

import os from urllib.parse import unquote, urlparse import pytest PATHS_FILE = os.environ.get("PATHS_FILE", "").strip() def _key(url_or_path): """Compare decoded paths; /dir/index.php and /dir/ are the same page.""" path = unquote(urlparse(url_or_path).path or "/") return path.removesuffix("index.php") if path.endswith("/index.php") else path def _load(path): if not path: return None with open(path, encoding="utf-8") as handle: return frozenset(_key(line.strip()) for line in handle if line.startswith("/")) SELECTED = _load(PATHS_FILE) def in_selection(url_or_path): return SELECTED is None or _key(url_or_path) in SELECTED def skip_if_nothing_selected(locs): """A page subset that leaves this test nothing to check skips it, not fails.""" if SELECTED is not None and not locs: pytest.skip(f"none of the {len(SELECTED)} paths in PATHS_FILE apply to this test", allow_module_level=True) def selection_label(): return "(all)" if SELECTED is None else f"{len(SELECTED)} paths from {PATHS_FILE}"

The test file container/tests/test_url_selection.py:

import importlib import pytest import url_selection @pytest.fixture(autouse=True) def _restore_module(): yield importlib.reload(url_selection) # runs after monkeypatch restored PATHS_FILE def _reload(monkeypatch, value): monkeypatch.setenv("PATHS_FILE", value) return importlib.reload(url_selection) def test_unset_paths_file_selects_everything(monkeypatch): selection = _reload(monkeypatch, "") assert selection.in_selection("https://www.aredel.com/any/page.php") assert selection.selection_label() == "(all)" selection.skip_if_nothing_selected([]) # no subset -> no skip def test_listed_paths_match_exactly_and_index_php_equals_directory(monkeypatch, tmp_path): listed = tmp_path / "paths.txt" listed.write_text("/about/\n/ru/qa/page.php\n# comment\nnot-a-path\n", encoding="utf-8") selection = _reload(monkeypatch, str(listed)) assert selection.in_selection("https://www.aredel.com/about/") assert selection.in_selection("/about/index.php") assert selection.in_selection("http://php-apache/ru/qa/page.php") assert not selection.in_selection("/ru/qa/") assert not selection.in_selection("/about/team/") assert selection.selection_label() == f"2 paths from {listed}" with pytest.raises(pytest.skip.Exception): selection.skip_if_nothing_selected([])

The differences from the tiny example are small but each has a reason:

Run the file from the container directory:

./tools/run-tests-venv.sh -- tests/test_url_selection.py -v -p no:cacheprovider

============================= test session starts ============================== platform linux -- Python 3.12.16, pytest-9.2.0, pluggy-1.7.0 -- … rootdir: <project> plugins: allure-pytest-2.16.2 collecting ... collected 2 items test_url_selection.py::test_unset_paths_file_selects_everything PASSED [ 50%] test_url_selection.py::test_listed_paths_match_exactly_and_index_php_equals_directory PASSED [100%] ============================== 2 passed in 0.09s ===============================

This run was made with pytest 9.2.0 on Python 3.12.16 from a scratch venv, which is why the run prints an installed allure-pytest plugin; the host venv script installs the same container/tests/requirements.txt and runs with its own .venv/bin/python. The -p no:cacheprovider is there because the checkout used for this run was mounted read-only and pytest could not create its cache directory; on the host it is not needed. The two test names are the two behaviours: no subset at all, and an exact subset that also proves the negative cases.

Article author: Arthur Isaev