Generated files without a post-commit hook
A file that a repository commits and that is built from its history, such as an article feed, is tempting to rebuild in a post-commit hook: the hook runs after every commit, writes the file and stages it. The first commits look fine. Merges, worktrees and the next commit break it. This page shows the break in a throwaway repository, then builds the same file from the first-parent history of main with a script that commits only that file.
Every command was run in a fresh repository with git 2.43.0 and Python 3.12.3. The output is pasted as it printed; commit hashes differ on your machine.
The problem: a hook that stages its output
Start with a repository and a local identity, so the commits need no global configuration. It lives in a folder named old:
mkdir old
cd old
git init -q -b main
git config user.name Demo
git config user.email demo@example.invalid
Add a page and an empty feed, then commit them:
mkdir pages
printf '<title>Home</title>\n' > pages/index.php
printf '[]\n' > feed.json
git add feed.json pages/index.php
git commit -m "Start the site"
[main (root-commit) 5c2ae92] Start the site 2 files changed, 2 insertions(+) create mode 100644 feed.json create mode 100644 pages/index.php
Make a branch with a page commit. It is made before the hook exists, so no hook ever runs for it:
git switch -q -c page-c
printf '<title>Page C</title>\n' > pages/c.php
git add pages/c.php
git commit -q -m "Add page C"
git switch -q main
Now the hook. It runs after every commit, appends the pages that commit added to feed.json, and stages the file. The file is written in place: opened with "w", so it is truncated first and then written again:
vi .git/hooks/post-commit
#!/usr/bin/env python3 # old way: add this commit's pages to feed.json in place, then stage it import json, subprocess pages = subprocess.run(["git", "diff-tree", "--no-commit-id", "-r", "--name-only", "--root", "HEAD", "--", "pages/"], capture_output=True, text=True).stdout.split() with open("feed.json") as f: items = json.load(f) items += [{"url": "/" + p} for p in pages] with open("feed.json", "w") as f: # truncate, then write json.dump(items, f, indent=2) f.write("\n") subprocess.run(["git", "add", "feed.json"], check=True)
chmod +x .git/hooks/post-commit
Add a page on main and look at the status:
printf '<title>Git hooks</title>\n' > pages/hooks.php
git add pages/hooks.php
git commit -m "Add the hooks page"
[main 41c4afa] Add the hooks page 1 file changed, 1 insertion(+) create mode 100644 pages/hooks.php
git status --short
M feed.json
The feed is staged, and nobody ran git add on it. The next commit is about something else:
echo notes > notes.txt
git add notes.txt
git commit -m "Add notes"
[main 2478a9d] Add notes 2 files changed, 6 insertions(+), 1 deletion(-) create mode 100644 notes.txt
git show --stat --oneline HEAD
2478a9d Add notes feed.json | 6 +++++- notes.txt | 1 + 2 files changed, 6 insertions(+), 1 deletion(-)
git show --stat proves it: the feed went into a commit about notes. Next, merge the branch from before the hook:
git merge --no-ff -m "Merge page C" page-c
Merge made by the 'ort' strategy. pages/c.php | 1 + 1 file changed, 1 insertion(+) create mode 100644 pages/c.php
git diff-tree -r --name-only HEAD | wc -l
0
cat feed.json
[ { "url": "/pages/hooks.php" } ]
git merge does not run post-commit, and git diff-tree -r prints nothing for a merge commit unless it gets -m. Page C is in the history and not in the feed, and no later run of the hook will look at it.
Last, a second worktree. Hooks live in .git/hooks, which every worktree of the repository shares, so the hook runs in the worktree too and edits that worktree's copy of the feed:
git worktree add -q ../old-wt -b wt-page
cd ../old-wt
printf '<title>Worktree page</title>\n' > pages/wt.php
git add pages/wt.php
git commit -m "Add the worktree page"
[wt-page 68e38f8] Add the worktree page 1 file changed, 1 insertion(+) create mode 100644 pages/wt.php
cat feed.json
[ { "url": "/pages/hooks.php" }, { "url": "/pages/wt.php" } ]
Back in the main worktree, make one more page commit:
cd ../old
printf '<title>Main page</title>\n' > pages/main.php
git add pages/main.php
git commit -m "Add the main page"
[main b257298] Add the main page 1 file changed, 1 insertion(+) create mode 100644 pages/main.php
git status --short
M feed.json
The main worktree has the feed staged again. Merging the worktree branch now is refused:
git merge wt-page
error: Your local changes to the following files would be overwritten by merge: feed.json Merge with strategy ort failed.
git worktree remove ../old-wt
fatal: '../old-wt' contains modified or untracked files, use --force to delete it
That is four problems in one run. The feed reached an unrelated commit. A merge posted nothing. Every worktree edited its own copy of the feed, and the staged copy blocked both the merge and the removal of the worktree. Nothing locks the file either: the hook opens it with "w" and writes in place, so two runs at once can write over each other.
The fix: build the file from history
The fix removes the hook and makes the feed a derived file, rebuilt by a script that runs when you say so. Five parts do the work: the file records the last commit it covers, the script walks only the commits after it, it diffs each commit with its first parent, it holds a lock shared by all worktrees, and it commits only its own file.
- Store the marker in the file. through is the last
commit of main that is already in the feed. An empty value means the first
run must name a starting commit with --from:
printf '{\n "through": "",\n "items": []\n}\n' > feed.json
- Write the script. It is standard library only, so there is nothing to
install. Save it as build_feed.py in the repository
root:
vi build_feed.py
#!/usr/bin/env python3 """Rebuild feed.json from the first-parent history of main, then commit only that file.""" import argparse, fcntl, json, os, re, subprocess, sys, tempfile FEED = "feed.json" def git(*args): return subprocess.run(["git", *args], check=True, text=True, capture_output=True).stdout.strip() def changed_pages(commit): """(action, path) of the pages a commit adds or changes, against its first parent.""" out = git("diff-tree", "-r", "--no-commit-id", "--name-status", "--find-renames=40%", f"{commit}^1", commit) rows = [line.split("\t") for line in out.splitlines()] return [("created" if r[0] == "A" else "modified", r[-1]) for r in rows if r[0][0] != "D" and r[-1].startswith("pages/") and r[-1].endswith(".php")] def page_title(path): match = re.search(r"<title>(.*?)</title>", open(path, encoding="utf-8").read()) return match.group(1) if match else path def main(): ap = argparse.ArgumentParser() ap.add_argument("--dry-run", action="store_true") ap.add_argument("--from", dest="start", help="first run: the commit already in the feed") ap.add_argument("--skip", action="append", default=[], help="leave this commit out") args = ap.parse_args() top = git("rev-parse", "--show-toplevel") os.chdir(top) lock_path = os.path.join(top, git("rev-parse", "--git-common-dir"), "feed.lock") lock = open(lock_path, "a") try: fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB) except BlockingIOError: sys.exit(f"another run holds {lock_path}") feed = json.load(open(FEED, encoding="utf-8")) start = args.start or feed["through"] if not start: sys.exit("feed.json has no through: pass --from <commit>") skip = {git("rev-parse", c) for c in args.skip} log = git("log", "--first-parent", "--reverse", "--format=%H %cs", f"{git('rev-parse', start)}..main") commits = [line.split() for line in log.splitlines()] if not commits: return print("nothing new") new = [] for sha, date in commits: if sha not in skip: new += [{"url": "/" + p, "title": page_title(p), "date": date, "action": a} for a, p in changed_pages(sha)] new.reverse() if args.dry_run: for item in new: print(item["date"], item["action"], item["url"]) return print(f"{len(new)} new, nothing written") feed["through"] = commits[-1][0] feed["items"] = new + feed["items"] fd, tmp = tempfile.mkstemp(dir=".", prefix=".feed-") with os.fdopen(fd, "w", encoding="utf-8") as handle: json.dump(feed, handle, indent=2, ensure_ascii=False) handle.write("\n") os.replace(tmp, FEED) subprocess.run(["git", "commit", "-m", f"Feed: {len(new)} new through {commits[-1][0][:9]}", "--", FEED], check=True) main()
- Read the script in pieces. changed_pages compares each commit with its first parent, so a merge reports the pages its branch brought in. --git-common-dir gives one lock file for every worktree. mkstemp with os.replace writes the new file in one step: a reader sees the old file or the new one, never half of it. The last line commits feed.json with a path, so anything else that is staged stays staged.
- Run it on main. The usual sequence is a dry run first, which prints the
entries and writes nothing, then the real run. Ordinary commits leave the feed
alone; only the script changes it.
python3 build_feed.py --dry-run
python3 build_feed.py
The fix at work
Repeat the same repository with the script. Start a new one in a folder named new, with the feed file from step 1 and the script from step 2:
cd ..
mkdir new
cd new
git init -q -b main
git config user.name Demo
git config user.email demo@example.invalid
mkdir pages
printf '<title>Home</title>\n' > pages/index.php
printf '{\n "through": "",\n "items": []\n}\n' > feed.json
vi build_feed.py
Commit the three files, and keep the root commit in a variable: the first run starts after it.
git add feed.json pages/index.php build_feed.py
git commit -q -m "Start the site"
ROOT=$(git rev-list --max-parents=0 main)
echo "root $ROOT"
root 849ebcbc36ca31f1d63adefdda01b7ebb17b77c4
Without a starting commit the script stops before it reads anything:
python3 build_feed.py; echo "exit $?"
feed.json has no through: pass --from <commit> exit 1
A page commit and an empty status: nothing is staged, the feed is not touched.
printf '<title>Git hooks</title>\n' > pages/hooks.php
git add pages/hooks.php
git commit -q -m "Add the hooks page"
git status --short
(no output)
Make a branch with a page, and merge it with --no-ff so that the merge is a commit of its own:
git switch -q -c page-c
printf '<title>Page C</title>\n' > pages/c.php
git add pages/c.php
git commit -q -m "Add page C"
git switch -q main
git merge -q --no-ff -m "Merge page C" page-c
A second worktree adds its page on its own branch, and main adds another one. Both feeds stay untouched until the script runs:
git worktree add -q ../new-wt -b wt-page
(cd ../new-wt && printf '<title>Worktree page</title>\n' > pages/wt.php && git add pages/wt.php && git commit -q -m "Add the worktree page")
printf '<title>Main page</title>\n' > pages/main.php
git add pages/main.php
git commit -q -m "Add the main page"
git merge -q --no-ff -m "Merge wt-page" wt-page
git log --oneline --first-parent
d12559e Merge wt-page 5994cc2 Add the main page 42c87bf Merge page C 845a672 Add the hooks page 849ebcb Start the site
The dry run walks the commits after $ROOT. Page C comes from the merge and the worktree page from its merge, dated by the commit:
python3 build_feed.py --dry-run --from "$ROOT"; echo "exit $?"
2026-11-16 created /pages/wt.php 2026-11-16 created /pages/main.php 2026-11-16 created /pages/c.php 2026-11-16 created /pages/hooks.php 4 new, nothing written exit 0
Now the real run. It writes both files and commits only feed.json:
python3 build_feed.py --from "$ROOT"; echo "exit $?"
[main a4fb61d] Feed: 4 new through d12559ed9 1 file changed, 27 insertions(+), 2 deletions(-) exit 0
git show --stat --oneline HEAD
a4fb61d Feed: 4 new through d12559ed9 feed.json | 29 +++++++++++++++++++++++++++-- 1 file changed, 27 insertions(+), 2 deletions(-)
Next a page is committed, and a file is staged by hand. The run that follows must not take the staged file along. It does not need --from now, because through is in the file:
printf '<title>Late page</title>\n' > pages/late.php
git add pages/late.php
git commit -q -m "Add the late page"
echo draft > draft.txt
git add draft.txt
python3 build_feed.py
[main aa4fa28] Feed: 1 new through a4ee94168 1 file changed, 7 insertions(+), 1 deletion(-)
git status --short
A draft.txt
git show --stat --oneline HEAD
aa4fa28 Feed: 1 new through a4ee94168 feed.json | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
draft.txt is still staged, and the commit holds only the feed. Next, a bulk edit that changes every page with no visible change. It is committed with a path, so only the pages go in:
for f in pages/*.php; do printf '<!-- reviewed -->\n' >> "$f"; done
git commit -q -m "Mark every page reviewed" -- pages
git rev-parse --short HEAD
7710c58
Without --skip, the dry run lists every page as changed:
python3 build_feed.py --dry-run
2026-11-16 modified /pages/wt.php 2026-11-16 modified /pages/main.php 2026-11-16 modified /pages/late.php 2026-11-16 modified /pages/index.php 2026-11-16 modified /pages/hooks.php 2026-11-16 modified /pages/c.php 6 new, nothing written
python3 build_feed.py --dry-run --skip 7710c58
0 new, nothing written
A real run with that skip commits the feed with the marker moved past the bulk commit, and nothing else:
python3 build_feed.py --skip 7710c58
[main d64673f] Feed: 0 new through 7710c58f7 1 file changed, 1 insertion(+), 1 deletion(-)
git show --stat --oneline HEAD
d64673f Feed: 0 new through 7710c58f7 feed.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
The lock is shared by every worktree. Hold it with flock for two seconds, and a dry run started meanwhile stops:
flock .git/feed.lock -c 'sleep 2' & sleep 0.5; python3 build_feed.py --dry-run; wait
another run holds …/article-demo/new/.git/feed.lock
The directory part of that path is cut here; the message printed it in full.
Variants and limits
The script takes three options:
- --dry-run prints the entries and writes nothing. Run it before every real run.
- --skip COMMIT leaves one commit out, for a bulk edit with no visible change. The marker still moves past it.
- --from COMMIT starts after a commit instead of through. Use it for the first run, or after the history was rewritten.
The limits matter as much as the options:
- The feed changes only when the script runs. A commit never updates it, so a page is in the feed after the next run, not after its commit.
- Titles come from the working tree. Run the script on main, where the working tree is main. The toy script does not check the branch; a fuller version refuses to run on any other branch, and refuses a feed file that differs from HEAD, so it never commits a hand edit by accident.
- The toy has no per-URL rules. A fuller version keeps one created line per URL, one modified line per URL per day, and a moved line for a renamed page, retires the old URL, skips the feed's own files, and splits the list at 500 entries into an archive file. It also refuses a --from that is not an ancestor of main.
- A history rewrite needs --from. The through commit may no longer be in main, so name the commit the feed already has.
The lesson of the page is the commit with paths: git commit -- feed.json takes only that file from the index. That is what lets a derived file share a repository with work that is still staged, which the hook could never allow.
The hook page has the rest of the picture: what a post-commit hook can and cannot do, and when a hook is the right tool. The post-commit hook explains it.
Article author: Klavdiya Kondakova