ShellCheck: finding bugs in bash scripts before they run
ShellCheck reads a bash or sh script without running it and reports the mistakes that make scripts break on the first file name with a space, the first empty variable or the first cd into a folder that is not there. Each finding has a code, such as SC2086, a severity and usually a suggested fix. This page runs it on a small script with typical mistakes, fixes them, silences a false alarm and adds the check to a pre-commit hook. Every command was run with ShellCheck 0.11.0 on Ubuntu 24.04; the output is pasted as it printed.
Running ShellCheck
Ubuntu and Debian have a package:
sudo apt install shellcheck
It lags behind, though: Ubuntu 24.04 offers 0.9.0. The image koalaman/shellcheck:stable from the authors has the current release and needs nothing but Docker. Its working folder is /mnt, so the current folder is mounted there:
docker run --rm koalaman/shellcheck:stable --version
ShellCheck - shell script analysis tool version: 0.11.0 license: GNU General Public License, version 3 website: https://www.shellcheck.net
With the package installed, every docker run ... below is just shellcheck followed by the same options and file names.
A first report
The script packs a folder into an archive with the date in its name. It works on the author's machine and has six kinds of mistakes:
vi backup.sh
#!/bin/bash # backup.sh: pack a folder into a dated archive src=$1 dest=/tmp/backups name=backup-`date +%F`.tar.gz unused="old value" cd $dest for f in $(ls $src); do echo "Found $f" done tar czf $name $src if [ $? == 0 ]; then echo Done: $dest/$name fi
A copy of this first version is kept for later sections:
cp backup.sh backup.orig
docker run --rm -v "$PWD:/mnt" koalaman/shellcheck:stable backup.sh
In backup.sh line 5: name=backup-`date +%F`.tar.gz ^--------^ SC2006 (style): Use $(...) notation instead of legacy backticks `...`. Did you mean: name=backup-$(date +%F).tar.gz In backup.sh line 6: unused="old value" ^----^ SC2034 (warning): unused appears unused. Verify use (or export if used externally). In backup.sh line 8: cd $dest ^------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails. Did you mean: cd $dest || exit In backup.sh line 9: for f in $(ls $src); do ^--------^ SC2045 (error): Iterating over ls output is fragile. Use globs. ^--^ SC2086 (info): Double quote to prevent globbing and word splitting. Did you mean: for f in $(ls "$src"); do In backup.sh line 12: tar czf $name $src ^---^ SC2086 (info): Double quote to prevent globbing and word splitting. ^--^ SC2086 (info): Double quote to prevent globbing and word splitting. Did you mean: tar czf "$name" "$src" In backup.sh line 13: if [ $? == 0 ]; then ^-- SC2181 (style): Check exit code directly with e.g. 'if mycmd;', not indirectly with $?. In backup.sh line 14: echo Done: $dest/$name ^---^ SC2086 (info): Double quote to prevent globbing and word splitting. Did you mean: echo Done: $dest/"$name" For more information: https://www.shellcheck.net/wiki/SC2045 -- Iterating over ls output is fragi... https://www.shellcheck.net/wiki/SC2034 -- unused appears unused. Verify use... https://www.shellcheck.net/wiki/SC2164 -- Use 'cd ... || exit' or 'cd ... |...
The exit status is 1 when there is at least one finding and 0 when there is none, which is what a hook or a CI job needs.
Reading the report
Each block names the line, quotes it, and marks the exact part with ^--^ under it. Then comes the code, the severity and the message, and for many findings a "Did you mean" line with the corrected line. The codes have a page each on the ShellCheck wiki, and the report lists the links for the most severe ones at the end. The four severities, from the most to the least serious:
| Severity | Meaning | Here |
|---|---|---|
| error | the script will misbehave | SC2045: a loop over the words of ls output breaks on names with spaces |
| warning | very likely a bug | SC2164: if cd fails, the script goes on in the wrong folder; SC2034: a variable that nothing reads |
| info | a risk worth fixing | SC2086: an unquoted variable is split into words and expanded as a glob |
| style | a better way to write it | SC2006: backticks; SC2181: testing $? instead of the command |
ShellCheck also follows values: $dest is not flagged on lines 8 and 14, because it always holds /tmp/backups, which has no space and no glob character. $src comes from the command line and can hold anything.
Fixing: the automatic part
-f diff prints the "Did you mean" fixes as a patch instead of a report:
docker run --rm -v "$PWD:/mnt" koalaman/shellcheck:stable -f diff backup.sh > fix.patch
cat fix.patch
--- a/backup.sh +++ b/backup.sh @@ -2,14 +2,14 @@ # backup.sh: pack a folder into a dated archive src=$1 dest=/tmp/backups -name=backup-`date +%F`.tar.gz +name=backup-$(date +%F).tar.gz unused="old value" -cd $dest -for f in $(ls $src); do +cd $dest || exit +for f in $(ls "$src"); do echo "Found $f" done -tar czf $name $src +tar czf "$name" "$src" if [ $? == 0 ]; then - echo Done: $dest/$name + echo Done: $dest/"$name" fi
The paths have the a/ and b/ prefixes of a git diff, so patch -p1 or git apply applies it:
patch -p1 < fix.patch
patching file backup.sh
-f gcc prints one line per finding, the format of GCC compiler errors that editors understand. Three findings are left:
docker run --rm -v "$PWD:/mnt" koalaman/shellcheck:stable -f gcc backup.sh
backup.sh:6:1: warning: unused appears unused. Verify use (or export if used externally). [SC2034] backup.sh:9:10: error: Iterating over ls output is fragile. Use globs. [SC2045] backup.sh:13:6: note: Check exit code directly with e.g. 'if mycmd;', not indirectly with $?. [SC2181]
Fixing: the part that needs a person
The three that are left have no mechanical fix: whether the variable is needed, how to list the files, which command to test. The unused variable goes, the loop uses a glob, which keeps names with spaces whole, and the if tests tar itself. The echo gets quotes around the whole message:
vi backup.sh
#!/bin/bash # backup.sh: pack a folder into a dated archive src=$1 dest=/tmp/backups name=backup-$(date +%F).tar.gz cd "$dest" || exit for f in "$src"/*; do echo "Found $f" done if tar czf "$name" "$src"; then echo "Done: $dest/$name" fi
A clean script prints nothing:
docker run --rm -v "$PWD:/mnt" koalaman/shellcheck:stable backup.sh
The exit status is 0.
Only what matters: -S and -f gcc
On an old script with hundreds of findings, start with the serious ones. -S warning (--severity) hides info and style. On the first version of the script, kept as backup.orig, three findings are left:
docker run --rm -v "$PWD:/mnt" koalaman/shellcheck:stable -S warning -f gcc backup.orig
backup.orig:6:1: warning: unused appears unused. Verify use (or export if used externally). [SC2034] backup.orig:8:1: warning: Use 'cd ... || exit' or 'cd ... || return' in case cd fails. [SC2164] backup.orig:9:10: error: Iterating over ls output is fragile. Use globs. [SC2045]
-S error keeps errors only; -S style, the default, shows everything.
Silencing a false alarm
ShellCheck does not follow every bash feature. A nameref, local -n into="$1", makes into another name for the variable whose name is in $1. Writing to into fills that variable, but ShellCheck sees an assignment that nothing reads:
printf 'one\ntwo\n' > a.txt && printf 'three\n' > b.txt
vi count.sh
#!/bin/bash # count.sh: line counts of files, kept in an associative array declare -A COUNT=() fill() { # $1: name of the array to fill, the rest: files local -n into="$1" local f for f in "${@:2}"; do into["$f"]=$(wc -l < "$f") done } fill COUNT a.txt b.txt for f in "${!COUNT[@]}"; do echo "$f ${COUNT[$f]}" done
docker run --rm -v "$PWD:/mnt" koalaman/shellcheck:stable count.sh
In count.sh line 8: into["$f"]=$(wc -l < "$f") ^--------^ SC2034 (warning): into appears unused. Verify use (or export if used externally). For more information: https://www.shellcheck.net/wiki/SC2034 -- into appears unused. Verify use (...
The script works:
bash count.sh
a.txt 2 b.txt 1
A directive on the line before the command turns one code off for that command only. The comment after it is for the next reader:
vi count.sh
for f in "${@:2}"; do # shellcheck disable=SC2034 # into is a nameref: this fills COUNT into["$f"]=$(wc -l < "$f") done
docker run --rm -v "$PWD:/mnt" koalaman/shellcheck:stable count.sh
No output, exit status 0. A directive placed right after the #! line covers the whole file. Prefer the narrow one, and prefer code without a directive when it is just as clear: here the function could also print its results and let the caller fill the array. The same SC2034 on a nameref once failed a commit that had passed a run of the hooks on the changed files; that story is in A linter that checked nothing.
Settings for a project: .shellcheckrc
Options that hold for every script of a project go into .shellcheckrc, in the folder of the script or in any folder above it. Each line is a long option without the dashes. A team that accepts $? tests turns SC2181 off for all scripts:
printf 'disable=SC2181\n' > .shellcheckrc
docker run --rm -v "$PWD:/mnt" koalaman/shellcheck:stable -S style -f gcc backup.orig
backup.orig:5:13: note: Use $(...) notation instead of legacy backticks `...`. [SC2006] backup.orig:6:1: warning: unused appears unused. Verify use (or export if used externally). [SC2034] backup.orig:8:1: warning: Use 'cd ... || exit' or 'cd ... || return' in case cd fails. [SC2164] backup.orig:9:10: error: Iterating over ls output is fragile. Use globs. [SC2045] backup.orig:9:15: note: Double quote to prevent globbing and word splitting. [SC2086] backup.orig:12:9: note: Double quote to prevent globbing and word splitting. [SC2086] backup.orig:12:15: note: Double quote to prevent globbing and word splitting. [SC2086] backup.orig:14:22: note: Double quote to prevent globbing and word splitting. [SC2086]
Line 13 is not in the list any more. Other useful lines are shell=bash for scripts without a #! line and external-sources=true to let ShellCheck read the files a script includes with source.
ShellCheck in pre-commit
The pre-commit framework can run the same image. language: docker_image takes an image name as the entry, mounts the repository and passes the staged files; types: [shell] selects shell scripts by extension and by the #! line:
git init -q demo && cd demo
git config user.email you@example.com && git config user.name You
vi .pre-commit-config.yaml
repos: - repo: local hooks: - id: shellcheck name: shellcheck language: docker_image entry: koalaman/shellcheck:stable types: [shell]
pre-commit install
pre-commit installed at .git/hooks/pre-commit
printf '#!/bin/bash\ncd $1\nls\n' > list.sh
git add . && git commit -qm "Adds list.sh"
shellcheck...............................................................Failed - hook id: shellcheck - exit code: 1 In list.sh line 2: cd $1 ^---^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails. ^-- SC2086 (info): Double quote to prevent globbing and word splitting. Did you mean: cd "$1" || exit For more information: https://www.shellcheck.net/wiki/SC2164 -- Use 'cd ... || exit' or 'cd ... |... https://www.shellcheck.net/wiki/SC2086 -- Double quote to prevent globbing ...
sed -i 's/^cd $1$/cd "$1" || exit/' list.sh && git add list.sh
git commit -qm "Adds list.sh"
shellcheck...............................................................Passed
Because the framework passes the file names, the hook also checks the files given to pre-commit run shellcheck --files .... A hook written with pass_filenames: false that picks the staged files itself does not: with nothing staged it checks nothing and passes.
What it does not catch
- Logic. A wrong path, a wrong option of tar or a wrong order of commands is a correct script to ShellCheck.
- What happens at run time. A file that is missing, a command that is not installed, a variable that is empty because the caller forgot it. Run the script, and add set -u to stop on unset variables.
- Code it cannot see. Text built at run time and passed to eval, and included files unless external-sources is on.
- Everything about the feature it misreads. Namerefs and some indirect assignments give false alarms; silence them one by one with a reason.
Article author: Klavdiya Kondakova