dotlinux guide

Shell Scripting: Patterns and AntiPatterns

Table of Contents

  1. Fundamentals of Shell Scripting
  2. Essential Shell Scripting Patterns
  3. Critical Shell Scripting AntiPatterns
  4. Common Practices vs. Best Practices
  5. Conclusion
  6. References

Fundamentals of Shell Scripting

A shell script is a text file containing a sequence of shell commands. It is executed by a shell interpreter (e.g., bash, sh, zsh). While shells are flexible, they prioritize brevity over safety, making it easy to write scripts that “work” but fail unexpectedly under edge cases.

Why Patterns and AntiPatterns Matter:

  • Patterns ensure scripts are reliable, readable, and maintainable.
  • AntiPatterns introduce silent failures, security risks, or unmaintainable code.

Before diving into specifics, let’s clarify: This blog focuses on bash (the most common shell), but many principles apply to POSIX-compliant shells like sh.

Essential Shell Scripting Patterns

1. Shebang Line: Specify the Shell Explicitly

The shebang (#!) at the top of a script tells the kernel which interpreter to use. Always specify it to avoid relying on the user’s default shell (which may vary).

Pattern: Use #!/usr/bin/env bash for portability (searches PATH for bash), or #!/bin/bash if the path is fixed. Avoid #!/bin/sh unless writing POSIX-compliant scripts (as sh may lack bash features).

Example:

#!/usr/bin/env bash
# This script uses bash-specific features (e.g., arrays, [[ ]])

2. Error Handling with set Options

By default, shells ignore errors in commands and continue execution. Use set options to enforce strictness.

Key Options:

  • -e: Exit immediately if any command fails (prevents silent failures).
  • -u: Treat unset variables as errors (avoids accidental use of undefined variables).
  • -o pipefail: Make a pipeline exit with the status of the last non-zero command (instead of the last command).

Pattern: Start scripts with set -euo pipefail to catch errors early.

Example:

#!/usr/bin/env bash
set -euo pipefail  # Strict error checking

# If "invalid_command" fails, the script exits immediately (thanks to -e)
invalid_command  # Script aborts here with an error
echo "This line will NOT run"

3. Variable Quoting: Prevent Word Splitting and Globbing

Shells split unquoted variables into words using IFS (Internal Field Separator) and expand globs (e.g., *). Quoting variables avoids this.

Pattern: Always quote variables with "$var" (double quotes) to preserve spaces and prevent globbing. Use 'single quotes' for literal strings.

Example:

#!/usr/bin/env bash
set -euo pipefail

filename="my file.txt"  # Contains a space

# Unquoted: Splits into "my" and "file.txt" → Error (file not found)
cat $filename  # Fails: "cat: my: No such file or directory"

# Quoted: Treated as a single argument → Works
cat "$filename"  # Success: Reads "my file.txt"

4. Idempotent Scripts: Safe to Run Multiple Times

An idempotent script produces the same result whether run once or multiple times. Avoid side effects like duplicate file creation or repeated configuration changes.

Pattern: Check for preconditions before acting (e.g., “create a directory only if it doesn’t exist”).

Examples:

  • Use mkdir -p to create directories (no error if they exist).
  • Check if a line exists in a config file before appending.
#!/usr/bin/env bash
set -euo pipefail

# Idempotent directory creation (no error if ~/myapp exists)
mkdir -p ~/myapp

# Append a line to /etc/profile only if it doesn't exist
LINE="export PATH=\$PATH:~/myapp/bin"
if ! grep -qxF "$LINE" /etc/profile; then
  echo "$LINE" >> /etc/profile
fi

5. Encapsulate Logic with Functions

Functions improve readability, reusability, and testability by grouping related code.

Pattern: Define functions for repetitive tasks, with clear names and error checking.

Example:

#!/usr/bin/env bash
set -euo pipefail

# Function to validate an email address (simplified example)
validate_email() {
  local email="$1"  # Local variable (avoids global pollution)
  if [[ ! "$email" =~ ^[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}$ ]]; then
    echo "Error: Invalid email address: $email" >&2  # Redirect to stderr
    return 1  # Return non-zero exit code to indicate failure
  fi
  echo "Valid email: $email"
  return 0
}

# Usage
validate_email "[email protected]"  # Success
validate_email "invalid-email"     # Fails and exits (due to set -e)

6. Check for Dependencies Before Execution

Scripts often rely on external tools (e.g., curl, jq). Failing to check if these tools exist leads to cryptic errors.

Pattern: Use command -v to check if a command exists before running it.

Example:

#!/usr/bin/env bash
set -euo pipefail

# Check if curl is installed
if ! command -v curl &> /dev/null; then
  echo "Error: 'curl' is required but not installed." >&2
  exit 1
fi

# Proceed only if curl exists
curl "https://example.com"

7. Secure Temporary Files with mktemp

Hardcoding temporary files (e.g., /tmp/myscript.tmp) risks race conditions (attacker creates the file first) or name collisions. Use mktemp to generate unique, secure temp files.

Pattern: Use mktemp to create temporary files/directories, and clean them up with trap (to handle script exits).

Example:

#!/usr/bin/env bash
set -euo pipefail

# Create a temporary file (auto-deleted on exit)
tempfile=$(mktemp)
trap 'rm -f "$tempfile"' EXIT  # Cleanup on script exit (even if it fails)

# Use the temp file
echo "Temporary data" > "$tempfile"
cat "$tempfile"

Critical Shell Scripting AntiPatterns

1. Omitting Error Handling

AntiPattern: Running commands without checking if they succeed. Silent failures can cascade, leading to data loss or incorrect behavior.

Example (Bad):

#!/bin/sh
# No error handling! Script continues even if "cp" fails.
cp important_file /backup/
rm important_file  # DANGER: If "cp" failed, this deletes the only copy!

Fix: Use set -euo pipefail and check exit codes explicitly:

#!/usr/bin/env bash
set -euo pipefail

cp important_file /backup/ || { echo "cp failed!" >&2; exit 1; }
rm important_file  # Only runs if "cp" succeeded

2. Unquoted Variables: A Recipe for Disaster

AntiPattern: Using unquoted variables, leading to word splitting, globbing, or accidental command injection.

Example (Bad):

#!/bin/sh
filename="file with spaces.txt"
rm $filename  # Splits into "rm file with spaces.txt" → Deletes "file", "with", "spaces.txt"!

Fix: Quote variables:

rm "$filename"  # Safely deletes "file with spaces.txt"

3. Hardcoding Paths and Assumptions

AntiPattern: Assuming tools or files exist at fixed paths (e.g., /usr/local/bin/python) or that the user has sudo access.

Example (Bad):

#!/bin/sh
# Assumes "python" is in /usr/bin (may not be true on all systems)
/usr/bin/python script.py

Fix: Use command -v to locate tools dynamically:

#!/usr/bin/env bash
set -euo pipefail

if ! command -v python &> /dev/null; then
  echo "python not found!" >&2; exit 1
fi
python script.py  # Uses the first "python" in PATH

4. Ignoring Command Exit Codes

AntiPattern: Ignoring $? (the exit code of the last command) or using || true to suppress errors blindly.

Example (Bad):

#!/bin/sh
# "grep" fails if pattern not found, but "|| true" hides the error
grep "pattern" file.txt || true
echo "Search complete!"  # Misleading: Script claims success even if "grep" failed

Fix: Check exit codes explicitly:

if grep "pattern" file.txt; then
  echo "Pattern found!"
else
  echo "Pattern not found." >&2
  exit 1  # Or handle gracefully
fi

5. Using Deprecated Syntax

AntiPattern: Relying on outdated syntax like backticks (`command`) instead of $(command), or [ ] instead of [[ ]] (for bash).

Example (Bad):

#!/bin/sh
# Backticks are harder to nest and less readable
date=`date +%F`
# [ ] lacks features like pattern matching and requires proper quoting
if [ $var = "value" ]; then ...

Fix: Use modern syntax:

date=$(date +%F)  # Easier to read and nest: $(command $(nested))
if [[ "$var" == "value" ]]; then ...  # Supports regex, globbing, and safer quoting

6. Overcomplicating with Shell

AntiPattern: Using shell for tasks better suited to languages like Python or Perl (e.g., parsing JSON, complex arithmetic, or regex-heavy text processing).

Example (Bad): Parsing JSON with grep/awk (fragile and error-prone):

#!/bin/sh
# Tries to extract "name" from JSON (fails if formatting changes)
curl https://api.example.com/user | grep -o '"name": "[^"]*"' | awk -F'"' '{print $4}'

Fix: Use jq (a JSON parser) or Python:

curl https://api.example.com/user | jq -r '.name'  # Reliable and simple

7. Insecure Temporary File Handling

AntiPattern: Using predictable temp file names (e.g., /tmp/myscript.$$, where $$ is the PID) or failing to clean up. Attackers can pre-create these files to overwrite data or execute code.

Example (Bad):

#!/bin/sh
tempfile="/tmp/myscript.$$"  # Predictable name!
echo "sensitive data" > "$tempfile"
# No cleanup: Leaves sensitive data in /tmp

Fix: Use mktemp and trap (as shown in the Patterns section).

Common Practices vs. Best Practices

Common PracticeBest PracticeRationale
#!/bin/sh#!/usr/bin/env bash (or explicit path)sh may lack bash features; env aids portability.
Unquoted variables ($var)Quoted variables ("$var")Prevents word splitting and globbing.
rm -rf /tmp/* to clean temp filesmktemp + trapAvoids deleting unintended files; secures temp data.
echo "Error"echo "Error" >&2Redirect errors to stderr (not stdout).
if [ $? -eq 0 ]if command; then ...More readable and avoids race conditions with $?.

Conclusion

Shell scripting is a powerful tool, but its flexibility demands discipline. By adopting patterns like strict error handling, variable quoting, and idempotency, you can write scripts that are reliable and maintainable. Conversely, avoiding antipatterns—such as unquoted variables, hardcoded paths, and insecure temp files—prevents bugs, security risks, and hair-pulling debugging sessions.

Remember: The best shell scripts are simple, explicit, and defensive. When in doubt, use tools like shellcheck (a static analysis tool for shell scripts) to catch issues early.

Happy scripting!

References