dotlinux guide

Top 10 Shell Scripting Mistakes and How to Avoid Them

Table of Contents

  1. Missing or Incorrect Shebang Line
  2. Ignoring Exit Codes
  3. Unquoted Variables
  4. Not Checking if Files/Directories Exist
  5. Overusing Global Variables
  6. Insecure Temporary Files
  7. Incorrect Use of set -e
  8. Not Escaping Special Characters in Input
  9. Poor Error Handling and Logging
  10. Not Testing Scripts Thoroughly

1. Missing or Incorrect Shebang Line

What’s the Problem?

The shebang line (#!/path/to/interpreter) tells the system which shell (e.g., bash, sh) to use to run the script. Omitting it or using an incorrect interpreter (e.g., #!/bin/sh when the script uses bash-specific features) leads to unexpected behavior or syntax errors.

Example of the Mistake

# No shebang line
echo "Hello, World"
array=("apple" "banana")  # Bash-specific syntax; fails if run with /bin/sh

Why It Fails

If the script is executed as ./script.sh without a shebang, the system defaults to the current shell (e.g., sh), which may not support bash features like arrays. Using #!/bin/sh (a POSIX-compliant shell) with bash-only code will also cause failures.

How to Fix It

Always start scripts with a explicit shebang. Use #!/bin/bash for bash scripts or #!/bin/sh only for POSIX-compliant code:

#!/bin/bash
echo "Hello, World"
array=("apple" "banana")  # Now works with bash

Best Practices

  • Use #!/usr/bin/env bash instead of #!/bin/bash for portability (works if bash is in the user’s PATH).
  • Avoid mixing bash and POSIX sh syntax.

2. Ignoring Exit Codes

What’s the Problem?

Commands return exit codes (0 = success, non-zero = failure). Scripts often ignore these, proceeding even if critical commands fail (e.g., cd to a non-existent directory).

Example of the Mistake

#!/bin/bash
cd /non/existent/directory  # Fails, but script continues
echo "Working in $(pwd)"    # Prints the original directory, not /non/existent/directory

Why It Fails

The cd command fails (exit code 1), but the script proceeds, leading to incorrect behavior (e.g., writing files to the wrong directory).

How to Fix It

Explicitly check exit codes or use set -e to exit on non-zero codes:

#!/bin/bash
set -e  # Exit immediately if any command fails

cd /non/existent/directory  # Script exits here
echo "Working in $(pwd)"    # Never runs

Or check manually for granular control:

#!/bin/bash
if ! cd /non/existent/directory; then
  echo "Error: Failed to cd to /non/existent/directory" >&2
  exit 1
fi
echo "Working in $(pwd)"

Best Practices

  • Use set -euo pipefail (see Section 9) for strict error handling.
  • Explicitly handle expected failures (e.g., grep "pattern" file || true to ignore “not found” errors).

3. Unquoted Variables

What’s the Problem?

Unquoted variables undergo word splitting and globbing, breaking filenames with spaces, tabs, or special characters (e.g., *).

Example of the Mistake

#!/bin/bash
file="my file.txt"  # Filename with a space
cat $file           # Unquoted: splits into "my" and "file.txt" → "cat: my: No such file"

Why It Fails

The variable $file expands to my file.txt, which the shell splits into two arguments (my and file.txt). cat tries to open my and file.txt separately, leading to errors.

How to Fix It

Always quote variables with "$var" to preserve spaces and prevent globbing:

#!/bin/bash
file="my file.txt"
cat "$file"  # Quoted: passes "my file.txt" as a single argument → works

Best Practices

  • Quote variables, command substitutions ("$(command)"), and array elements.
  • Avoid unquoted variables unless intentional word splitting is needed.

4. Not Checking if Files/Directories Exist

What’s the Problem?

Scripts often assume files or directories exist, leading to “No such file or directory” errors when they don’t.

Example of the Mistake

#!/bin/bash
input_file="$1"
grep "error" "$input_file" > errors.log  # Fails if $input_file doesn't exist

Why It Fails

If $1 is not provided or the file doesn’t exist, grep throws an error, and the script may write to errors.log incorrectly.

How to Fix It

Check for file/directory existence before using them:

#!/bin/bash
input_file="$1"

# Check if input file exists
if [ ! -f "$input_file" ]; then
  echo "Error: File $input_file not found." >&2
  exit 1
fi

grep "error" "$input_file" > errors.log

Best Practices

  • Use -f (file), -d (directory), -r (readable), or -x (executable) flags in [ ] or [[ ]] to validate paths.
  • Handle missing arguments with if [ $# -eq 0 ]; then echo "Usage: $0 <file>"; exit 1; fi.

5. Overusing Global Variables

What’s the Problem?

Global variables are accessible everywhere, leading to unintended side effects (e.g., functions modifying variables used elsewhere).

Example of the Mistake

#!/bin/bash
count=0

increment() {
  count=$((count + 1))  # Modifies global variable
}

increment
echo "Count: $count"  # Output: Count: 1 (expected)

# Later in the script...
count=5
increment
echo "Count: $count"  # Output: Count: 6 (unintended if 'count' was reused)

Why It Fails

The increment function relies on and modifies the global count, making the script harder to debug and prone to side effects.

How to Fix It

Use local variables in functions to limit scope:

#!/bin/bash
count=0

increment() {
  local local_count=$1  # Local variable; no side effects
  echo $((local_count + 1))
}

count=$(increment "$count")
echo "Count: $count"  # Output: Count: 1

count=5
count=$(increment "$count")
echo "Count: $count"  # Output: Count: 6 (explicit and intentional)

Best Practices

  • Pass variables as function arguments and return values explicitly.
  • Use local for all function variables to avoid polluting the global namespace.

6. Insecure Temporary Files

What’s the Problem?

Hardcoding temporary file paths (e.g., /tmp/myscript.tmp) creates security risks: attackers can predict filenames and use symlinks to overwrite sensitive data.

Example of the Mistake

#!/bin/bash
temp_file="/tmp/myscript.tmp"  # Predictable path
echo "Sensitive data" > "$temp_file"  # Risk of symlink attack

Why It Fails

An attacker could create a symlink /tmp/myscript.tmp pointing to /etc/passwd, causing the script to overwrite system files.

How to Fix It

Use mktemp to generate unique, secure temporary files:

#!/bin/bash
temp_file=$(mktemp)  # Creates a unique file (e.g., /tmp/tmp.abc123)
trap 'rm -f "$temp_file"' EXIT  # Clean up on exit

echo "Sensitive data" > "$temp_file"
# ... use $temp_file ...

Best Practices

  • Always use mktemp or mktemp -d for directories.
  • Add a trap to delete temporary files on script exit (even if it fails).

7. Incorrect Use of set -e

What’s the Problem?

set -e exits the script on any non-zero exit code, but some commands return non-zero intentionally (e.g., grep not finding a pattern, diff finding differences).

Example of the Mistake

#!/bin/bash
set -e

grep "pattern" file.txt  # Exits if "pattern" not found (non-zero exit code)
echo "This line never runs if grep fails"

Why It Fails

grep returns 1 if no matches are found, causing set -e to exit the script prematurely.

How to Fix It

Either disable set -e for specific commands or handle non-zero codes explicitly:

#!/bin/bash
set -e

# Option 1: Ignore non-zero with '|| true'
grep "pattern" file.txt || true  # Script continues even if grep fails

# Option 2: Check explicitly
if grep "pattern" file.txt; then
  echo "Pattern found"
else
  echo "Pattern not found"  # No exit
fi

Best Practices

  • Use set -e judiciously. Combine with set -o pipefail to exit on pipeline failures.
  • For commands with expected non-zero codes, add || true or check with if.

8. Not Escaping Special Characters in Input

What’s the Problem?

User input with special characters (e.g., ;, *, $) can be interpreted as shell commands, leading to injection attacks.

Example of the Mistake

#!/bin/bash
read -p "Enter a filename: " filename
cat "$filename"  #看似安全，但如果输入是 "; rm -rf /" 呢？

Why It Fails

If a user enters ; rm -rf /, the script executes cat; rm -rf /, deleting system files (in a worst-case scenario).

How to Fix It

Treat input as data, not code. Use printf "%q" to escape special characters or avoid passing input directly to commands:

#!/bin/bash
read -p "Enter a filename: " filename

# Validate input: check if it's a valid file path
if [[ ! "$filename" =~ ^[a-zA-Z0-9_./-]+$ ]]; then
  echo "Error: Invalid filename." >&2
  exit 1
fi

if [ -f "$filename" ]; then
  cat "$filename"
else
  echo "File not found." >&2
fi

Best Practices

  • Sanitize all user input (e.g., restrict allowed characters).
  • Avoid using eval with untrusted input.

9. Poor Error Handling and Logging

What’s the Problem?

Scripts often fail silently or lack context about errors (e.g., “Permission denied” without line numbers or file paths).

Example of the Mistake

#!/bin/bash
cp file1.txt /backup/  # Fails if /backup/ is read-only; no error message

Why It Fails

The cp command fails, but the script gives no indication, leaving the user unaware of the problem.

How to Fix It

Enable “bash strict mode” and add logging:

#!/bin/bash
set -o errexit  # Exit on non-zero
set -o nounset  # Treat unset variables as errors
set -o pipefail # Exit if any command in a pipeline fails
set -o xtrace   # Optional: Print commands as they run (debugging)

# Log errors with context
error_exit() {
  echo "Error: $1 (Line $2)" >&2
  exit 1
}

cp file1.txt /backup/ || error_exit "Failed to copy file1.txt to /backup/" "$LINENO"

Best Practices

  • Use set -euo pipefail for strict error checking.
  • Define an error-handling function with $LINENO to report where failures occur.

10. Not Testing Scripts Thoroughly

What’s the Problem?

Scripts are often deployed without testing edge cases (e.g., empty inputs, missing files, special characters), leading to production failures.

Example of the Mistake

#!/bin/bash
# Script to process a file, but never tested with empty files or spaces in filenames
file="$1"
grep "data" "$file" | wc -l

Why It Fails

  • If $1 is empty (nounset not enabled), the script runs grep "data" "", which errors.
  • If the file has spaces and $file is unquoted, word splitting occurs.

How to Fix It

Test rigorously with tools and edge cases:

  1. Use shellcheck: A linter for shell scripts (catches unquoted variables, missing checks, etc.). 
    shellcheck script.sh  # Highlights issues like unquoted variables
  2. Test Edge Cases:
    • Empty inputs (./script.sh "").
    • Files with spaces, special characters (./script.sh "my file.txt").
    • Missing dependencies (e.g., grep not installed).
  3. Debug with set -x: Print commands as they execute to trace issues.

Conclusion

Shell scripting is a powerful tool, but its flexibility comes with pitfalls. By avoiding these 10 common mistakes—using explicit shebangs, checking exit codes, quoting variables, securing temporary files, and testing rigorously—you can write scripts that are reliable, secure, and easy to maintain.

Adopt best practices like “bash strict mode” (set -euo pipefail), use shellcheck for linting, and always validate inputs. With these habits, you’ll transform error-prone scripts into robust automation tools.

References