dotlinux guide

Harnessing the Power of Unix and Linux Shell Scripting

Table of Contents

Fundamentals of Shell Scripting

What is Shell Scripting?

A shell script is a text file containing a sequence of commands executed by a Unix/Linux shell (e.g., Bash, Zsh). It automates repetitive tasks, combines system utilities, and extends functionality beyond single commands.

Types of Shells

The choice of shell impacts scripting syntax and features. Common shells include:

  • Bash (Bourne-Again Shell): Default on most Linux systems; rich in features (e.g., arrays, functions).
  • Zsh: Extends Bash with improvements like better autocompletion and themes.
  • Ksh (Korn Shell): POSIX-compliant with advanced scripting capabilities.
  • Sh (Bourne Shell): Legacy shell; minimal features, used for POSIX-portable scripts.

Bash is the most widely used, so we’ll focus on it here.

Basic Syntax

Shebang Line

The first line of a script specifies the interpreter:

#!/bin/bash  # Use Bash to execute the script

Variables

Store data for reuse. Declare without spaces; access with $:

NAME="Alice"
echo "Hello, $NAME!"  # Output: Hello, Alice!

Control Structures

If-Else: Conditional execution:

FILE="data.txt"
if [ -f "$FILE" ]; then  # Check if file exists
  echo "$FILE exists."
else
  echo "$FILE not found."
fi

Loops: Iterate over data:

  • for loop: 
    for i in {1..5}; do
  echo "Count: $i"
done
  • while loop: 
    COUNT=1
while [ $COUNT -le 3 ]; do
  echo "Loop $COUNT"
  ((COUNT++))  # Increment count
done

Command Substitution

Capture command output into a variable with $() or backticks:

TODAY=$(date +%Y-%m-%d)  # Store current date
echo "Today is $TODAY"

Usage Methods: Writing, Executing, and Debugging

Writing Scripts

Use a text editor (e.g., nano, vim, VS Code) to create .sh files:

nano hello.sh  # Open in nano; paste the script below

Example hello.sh:

#!/bin/bash
echo "Hello, Shell Scripting!"

Executing Scripts

  1. Make the script executable: 
    chmod +x hello.sh
  2. Run it: 
    ./hello.sh  # Executes in current shell
# Or: bash hello.sh (explicitly use Bash)

Debugging Techniques

  • Enable debugging: Add set -x to trace commands: 
    #!/bin/bash
set -x  # Debug mode on
NAME="Alice"
echo "Hello $NAME"
set +x  # Debug mode off
  • Exit on error: set -e stops execution if any command fails: 
    #!/bin/bash
set -e  # Exit on error
cp file1.txt /nonexistent/dir  # Fails; script exits here
echo "This won't run"
  • Linting: Use shellcheck to catch syntax errors: 
    shellcheck hello.sh  # Install with: sudo apt install shellcheck

Common Practices and Use Cases

Automation

Shell scripts shine for automating repetitive tasks.

Cron Jobs: Schedule scripts with cron (e.g., daily backups):

  1. Edit crontab: crontab -e
  2. Add: 0 2 * * * /home/user/backup.sh # Run daily at 2 AM

Backup Script:

#!/bin/bash
set -euo pipefail  # Strict error handling
BACKUP_DIR="/backups"
SOURCE="/home/user/docs"
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
tar -czf "$BACKUP_DIR/backup_$TIMESTAMP.tar.gz" "$SOURCE"
echo "Backup created: $BACKUP_DIR/backup_$TIMESTAMP.tar.gz"

System Monitoring

Track system health with scripts.

Disk Space Alert:

#!/bin/bash
THRESHOLD=90  # 90% usage
df -P / | awk 'NR==2 {print $5}' | sed 's/%//' | while read -r USAGE; do
  if [ "$USAGE" -ge "$THRESHOLD" ]; then
    echo "ALERT: Root partition at $USAGE% usage!" | mail -s "Disk Full" [email protected]
  fi
done

File Management

Process files efficiently with command-line tools.

Batch Rename Files:

#!/bin/bash
# Rename all .txt files to .md
for file in *.txt; do
  mv -- "$file" "${file%.txt}.md"  # ${file%.txt} removes .txt suffix
done

Search and Process Logs:

#!/bin/bash
# Count ERROR entries in /var/log/syslog
grep -c "ERROR" /var/log/syslog

Best Practices for Robust Scripts

Script Structure

  • Shebang: Always start with #!/bin/bash (or #!/bin/sh for POSIX).
  • Comments: Explain non-obvious logic: 
    #!/bin/bash
# Purpose: Backup user documents
# Usage: ./backup.sh <source_dir>
  • Cleanup: Use trap to handle signals (e.g., Ctrl+C): 
    trap 'echo "Script interrupted!"; exit 1' SIGINT SIGTERM

Error Handling

  • Check Exit Codes: Use $? to verify command success (0 = success): 
    cp file1.txt dest/
if [ $? -ne 0 ]; then  # If cp failed
  echo "Copy failed!" >&2  # >&2 redirects to stderr
  exit 1
fi
  • Strict Mode: set -euo pipefail catches errors early:
    • -e: Exit on error.
    • -u: Treat unset variables as errors.
    • -o pipefail: Exit if any command in a pipe fails.

Security

  • Avoid Hard-Coded Secrets: Use environment variables instead: 
    API_KEY="$MY_SECRET_KEY"  # Set via export MY_SECRET_KEY="..."
  • Restrict Permissions: Make scripts readable only by the owner: 
    chmod 700 sensitive_script.sh

Portability and Performance

  • Use POSIX Syntax for scripts running on multiple shells (e.g., avoid Bash arrays if targeting sh).
  • Minimize Subshells: Use { ... } instead of (...) for grouping to avoid spawning subshells: 
    { echo "Line 1"; echo "Line 2"; } > output.txt  # Faster than subshell

Advanced Techniques

Pipes and Filters

Combine commands with | to process data sequentially:

# Find large files (>100MB) and sort by size
find /home -type f -size +100M -exec du -h {} + | sort -rh

Functions and Arrays

Functions: Reuse code blocks:

greet() {
  local name="$1"  # Local variable
  echo "Hello, $name!"
}
greet "Bob"  # Output: Hello, Bob!

Arrays: Store lists of values:

FRUITS=("apple" "banana" "cherry")
echo "First fruit: ${FRUITS[0]}"  # apple
for fruit in "${FRUITS[@]}"; do
  echo "Fruit: $fruit"
done

Process Substitution

Treat command output as a file with <(command):

# Compare two command outputs
diff <(ls dir1) <(ls dir2)  # Shows differences between dir1 and dir2 listings

Conclusion

Unix/Linux shell scripting is a versatile skill that bridges manual command-line work and automation. By mastering fundamentals, adopting best practices, and leveraging advanced techniques, you can build scripts that are efficient, maintainable, and secure.

Start small—automate a daily task, then gradually tackle complex workflows. With practice, shell scripting will become an indispensable tool in your technical toolkit.

References