dotlinux guide

Shell Scripting Best Practices for Security and Performance

Table of Contents

  1. Why Shell Scripting Best Practices Matter
  2. Security Best Practices
  3. Performance Best Practices
  4. Common Pitfalls and How to Avoid Them
  5. Advanced Tips: Testing and Linting
  6. Conclusion
  7. References

Why Shell Scripting Best Practices Matter

  • Security Risks: Unvalidated input, unsafe command execution, and loose file permissions can expose systems to attacks like command injection, privilege escalation, or data exfiltration.
  • Performance Bottlenecks: Inefficient loops, excessive subshells, and over-reliance on external commands can slow down scripts, especially when processing large datasets or running in production.
  • Maintainability: Well-structured scripts with clear error handling are easier to debug, update, and extend.

Security Best Practices

1. Use a Strict Shebang Line

The shebang line (#!/path/to/shell) specifies the interpreter for the script. Ambiguous or incorrect shebangs can lead to unexpected behavior or security risks (e.g., using #!/bin/sh when relying on bash-specific features).

Bad Practice:

#!/bin/sh  # May not support bash features like arrays or [[ ]]

Good Practice:
Explicitly specify the shell and enable strict mode (covered next):

#!/bin/bash -euo pipefail
  • -e: Exit immediately if any command fails.
  • -u: Treat unset variables as errors.
  • -o pipefail: Make pipelines fail if any command in the pipeline fails.

2. Enable Strict Error Checking

Without strict error checking, scripts may continue executing even after critical failures, leading to silent data corruption or security gaps.

Bad Practice:

# Script continues even if "cd /nonexistent" fails
cd /nonexistent
rm -rf *  # Accidentally deletes files in current directory!

Good Practice:
Use set -euo pipefail (or include it in the shebang) to enforce strictness:

#!/bin/bash -euo pipefail

cd /nonexistent  # Script exits here due to -e
rm -rf *         # Never executed

3. Validate and Sanitize Input

Untrusted input (e.g., user arguments, environment variables, or file content) is a leading cause of shell script vulnerabilities. Always validate and sanitize inputs before use.

Validate Input Existence and Type

Bad Practice:

#!/bin/bash
filename="$1"
rm "$filename"  # If $1 is empty, deletes "rm *" (if unquoted) or fails silently

Good Practice:
Check for required arguments, file existence, and type:

#!/bin/bash -euo pipefail

# Check if argument is provided
if [ $# -ne 1 ]; then
  echo "Usage: $0 <filename>" >&2
  exit 1
fi

filename="$1"

# Validate file exists and is a regular file
if [ ! -f "$filename" ]; then
  echo "Error: $filename is not a valid file" >&2
  exit 1
fi

rm "$filename"

Sanitize Variables

Unsanitized variables can lead to word splitting, globbing, or command injection. Use parameter expansion and quoting to sanitize.

Bad Practice:

user_input="$1"
grep "$user_input" /etc/passwd  # Risky if $user_input contains special chars (e.g., "root; rm -rf /")

Good Practice:
Sanitize with regex or restrict allowed characters:

user_input="$1"

# Allow only alphanumeric and underscores
if [[ ! "$user_input" =~ ^[a-zA-Z0-9_]+$ ]]; then
  echo "Error: Invalid input. Only letters, numbers, and underscores allowed." >&2
  exit 1
fi

grep -- "$user_input" /etc/passwd  # -- prevents filename expansion if $user_input starts with "-"

4. Avoid Unsafe Functions and Constructs

Avoid eval

The eval command executes arbitrary code, making it极易 exploited with untrusted input.

Dangerous:

user_input="$1"
eval "echo $user_input"  # If $user_input is "hello; rm -rf /", this deletes files!

Alternative: Use parameter expansion or arrays instead:

user_input="$1"
echo "$user_input"  # No eval needed!

Avoid Unquoted Variables

Unquoted variables undergo word splitting and globbing, leading to unexpected behavior.

Bad Practice:

files="*.txt"
rm $files  # Expands to "rm file1.txt file2.txt" (okay here, but risky with spaces in filenames)

Good Practice:
Quote variables to preserve whitespace and prevent globbing:

files="*.txt"
rm "$files"  # Fails safely if $files has spaces (e.g., "my file.txt")

5. Restrict File Permissions

Overly permissive scripts or files can be modified by attackers. Set restrictive permissions for scripts and output files.

Bad Practice:

chmod 777 script.sh  # World-writable: anyone can modify the script!

Good Practice:

  • Scripts: Use chmod 700 (only owner can execute) or 755 (owner execute, group/others read).
  • Output files: Use umask 077 to restrict permissions to the owner only.
# Set umask for new files (restrict to owner read/write)
umask 077

# Create a sensitive file
echo "secret data" > sensitive.txt  # Permissions: -rw-------

6. Never Hardcode Secrets

Hardcoding passwords, API keys, or tokens in scripts is a critical security risk (e.g., exposed in version control).

Bad Practice:

#!/bin/bash
DB_PASSWORD="mypassword123"  # Hardcoded secret!
mysql -u root -p"$DB_PASSWORD"  # Secrets leak in process lists (ps aux)

Good Practice:

  • Use environment variables: DB_PASSWORD="$MYSQL_PASSWORD".
  • Read from secure files (e.g., ~/.ssh/id_rsa with chmod 600).
  • Use tools like vault or aws secretsmanager for dynamic secrets.

Performance Best Practices

1. Minimize Subshells

Subshells (created by $(...), (...), or pipes) add overhead. Avoid unnecessary subshells.

Bad Practice:

# Subshell for each line (slow for large files)
for line in $(cat large_file.txt); do
  echo "$line"
done

Good Practice:
Use a while loop with input redirection (no subshell):

while IFS= read -r line; do
  echo "$line"
done < large_file.txt

2. Optimize Loops

Shell loops are slow for large datasets. Use awk, sed, or grep for text processing instead of looping in shell.

Bad Practice:

# Slow: loops through each line to count words
word_count=0
while IFS= read -r line; do
  word_count=$((word_count + $(echo "$line" | wc -w)))
done < document.txt
echo "Total words: $word_count"

Good Practice:
Use awk for a single-pass solution:

word_count=$(awk '{total += NF} END {print total}' document.txt)
echo "Total words: $word_count"

3. Use Built-in Commands

External commands (e.g., grep, sed) are slower than shell built-ins. Use bash built-ins where possible.

Bad Practice:

# Uses external `grep` command
if echo "$var" | grep -q "pattern"; then
  echo "Match found"
fi

Good Practice:
Use [[ ]] (bash built-in) for pattern matching:

if [[ "$var" =~ "pattern" ]]; then
  echo "Match found"
fi

4. Leverage Parallel Processing

For CPU-bound tasks (e.g., processing multiple files), use xargs -P or GNU Parallel to parallelize work.

Example with xargs -P:

# Process 4 files in parallel
find ./logs -name "*.log" -print0 | xargs -0 -n 1 -P 4 gzip  # -P 4 = 4 parallel processes

Common Pitfalls and How to Avoid Them

PitfallRiskSolution
Unquoted variablesWord splitting, globbingAlways quote variables: "$var"
Ignoring exit codesSilent failuresUse set -e or check $? explicitly
Overusing cdScript breaks if directory missingUse absolute paths or `cd …
Wildcard expansionAccidental file deletionUse find with -exec instead of rm *

Advanced Tips: Testing and Linting

  • Lint with shellcheck: A static analysis tool to detect bugs and security issues. 
    shellcheck script.sh  # Highlights unquoted variables, missing checks, etc.
  • Test with bats: A testing framework for shell scripts (e.g., unit tests for functions).
  • Version Control: Store scripts in Git with descriptive commit messages for traceability.

Conclusion

Shell scripting is powerful, but its flexibility can lead to insecure or inefficient code. By adopting strict error checking, validating input, restricting permissions, and optimizing performance (e.g., minimizing subshells, using built-ins), you can write scripts that are secure, fast, and maintainable.

Always prioritize security (validate inputs, avoid hardcoded secrets) and measure performance (profile with time or perf) to identify bottlenecks. Tools like shellcheck and bats will help catch issues early in development.

References