dotlinux guide

Delving into Linux Shell Scripting: A Command Line Guide

In the realm of Linux and Unix-like systems, the command line is a powerful interface that enables users to interact with the operating system efficiently. At the heart of this interaction lies shell scripting—the art of writing sequences of commands in a script file to automate repetitive tasks, streamline workflows, and manage system operations. Whether you’re a system administrator automating backups, a developer deploying applications, or a hobbyist simplifying daily tasks, mastering shell scripting unlocks the full potential of the Linux command line. This guide will take you from the fundamentals of shell scripting to advanced techniques, common practices, and best practices. By the end, you’ll be equipped to write robust, efficient, and maintainable scripts that solve real-world problems.

Table of Contents

  1. Fundamentals of Linux Shell Scripting

    • What is a Shell?
    • Types of Shells
    • The Shebang Line
    • Your First Script: “Hello World”

  2. Basic Syntax and Core Concepts

    • Variables: Declaring and Using
    • Input/Output (I/O) Redirection
    • Control Structures: If-Else, Loops
    • Command Substitution

  3. Advanced Shell Scripting Techniques

    • Functions: Modular Code
    • Arrays: Managing Collections of Data
    • Error Handling and Debugging
    • Signal Handling with trap

  4. Common Practices for Writing Shell Scripts

    • Script Organization and Documentation
    • Modular Design with Functions
    • Commenting and Readability
    • Version Control for Scripts

  5. Best Practices for Robust Scripts

    • Security: Input Validation and Sanitization
    • Performance Optimization
    • Portability Across Shells
    • Testing and Debugging Tools

  6. Real-World Examples

    • Example 1: Automated Backup Script
    • Example 2: System Monitoring and Alerting

  7. Conclusion

  8. References

1. Fundamentals of Linux Shell Scripting

What is a Shell?

A shell is a command-line interpreter that acts as an interface between the user and the operating system kernel. It reads user input (commands), executes them, and returns output. Shells also support scripting: writing a sequence of commands in a file (a “script”) that the shell can execute as a single program.

Types of Shells

Linux supports multiple shells, each with unique features. The most common include:

  • Bash (Bourne-Again SHell): The default shell for most Linux distributions. It extends the original Bourne Shell (sh) with features like command history, tab completion, and scripting capabilities.
  • Zsh (Z Shell): A modern shell with advanced features like improved globbing, themes, and plugins (popular via frameworks like Oh My Zsh).
  • Fish (Friendly Interactive SHell): Designed for simplicity and user-friendliness, with syntax highlighting and auto-suggestions.

This guide focuses on Bash, as it is the most widely used and portable.

The Shebang Line

Every shell script starts with a “shebang” line, which tells the system which interpreter to use to execute the script. For Bash scripts, the shebang line is:

#!/bin/bash

Without this line, the system may use the default shell (e.g., sh), which lacks Bash-specific features.

Your First Script: “Hello World”

Let’s create a simple script to print “Hello, World!” to the terminal.

  1. Create a file named hello_world.sh using a text editor (e.g., nano or vim):

    #!/bin/bash
echo "Hello, World!"

  2. Make the script executable with the chmod command:

    chmod +x hello_world.sh

  3. Run the script:

    ./hello_world.sh

Output:

Hello, World!

Congratulations! You’ve written and executed your first shell script.

2. Basic Syntax and Core Concepts

Variables: Declaring and Using

Variables store data for later use. In Bash, declare variables without spaces around the = sign:

name="Alice"  # String variable
age=30        # Numeric variable

To access a variable, prefix it with $:

echo "Name: $name"   # Output: Name: Alice
echo "Age: $age"     # Output: Age: 30

Best Practice: Enclose variables in double quotes ("$name") to handle spaces in values. Use single quotes ('$name') only if you want to print the literal $name.

Input/Output (I/O) Redirection

Shell scripts often interact with files and the terminal. Key I/O redirection operators:

  • >: Redirects output to a file (overwrites existing content).
  • >>: Appends output to a file.
  • <: Reads input from a file.
  • 2>: Redirects error output (stderr) to a file.

Example:

# Write "Hello" to a file (overwrites if it exists)
echo "Hello" > output.txt

# Append "World" to the file
echo "World" >> output.txt

# Read input from output.txt and print
cat < output.txt  # Output: Hello\nWorld

Control Structures: If-Else, Loops

Control structures enable conditional execution and repetition.

If-Else Statements

Check conditions (e.g., file existence, numeric comparisons) with if-else:

#!/bin/bash
file="example.txt"

if [ -f "$file" ]; then  # -f checks if the file exists and is a regular file
    echo "$file exists."
elif [ -d "$file" ]; then  # -d checks if it's a directory
    echo "$file is a directory."
else
    echo "$file does not exist."
fi

Loops

  • For Loops: Iterate over a list of items (e.g., files, numbers):

    # Loop over numbers 1 to 5
for i in {1..5}; do
    echo "Number: $i"
done

# Loop over files in the current directory
for file in *.txt; do
    echo "Found text file: $file"
done

  • While Loops: Repeat until a condition is false:

    count=1
while [ $count -le 5 ]; do  # -le = "less than or equal to"
    echo "Count: $count"
    count=$((count + 1))  # Increment count
done

Command Substitution

Capture the output of a command and store it in a variable using $(command) or backticks (`command`):

# Get current date
current_date=$(date +%Y-%m-%d)
echo "Today is $current_date"  # Output: Today is 2024-05-20

# Count number of text files
txt_count=$(ls *.txt | wc -l)
echo "Text files: $txt_count"

3. Advanced Shell Scripting Techniques

Functions: Modular Code

Functions group reusable code into named blocks, improving readability and maintainability.

Syntax:

function_name() {
    # Code here
    echo "Hello from function"
}

Example with Parameters:

greet() {
    local name=$1  # $1 = first parameter, $2 = second, etc.
    echo "Hello, $name!"
}

greet "Bob"  # Output: Hello, Bob!

Return Values: Use return to exit a function and return an exit code (0 for success, non-zero for failure). To return a value, capture output with command substitution:

add() {
    local a=$1
    local b=$2
    echo $((a + b))  # Echo the result to return it
}

sum=$(add 5 3)
echo "Sum: $sum"  # Output: Sum: 8

Arrays: Managing Collections of Data

Arrays store multiple values in a single variable.

Declaration and Access:

fruits=("apple" "banana" "cherry")  # Declare array
echo "First fruit: ${fruits[0]}"     # Access element (0-based index)
echo "All fruits: ${fruits[@]}"      # Access all elements

Loop Through an Array:

for fruit in "${fruits[@]}"; do
    echo "Fruit: $fruit"
done

Error Handling and Debugging

Robust scripts handle errors gracefully. Key techniques:

  • Exit on Error: Use set -e to make the script exit immediately if any command fails:

    #!/bin/bash
set -e  # Exit on any error
cp file1.txt /nonexistent/dir  # Fails, so script exits here
echo "This line won't run"

  • Exit on Undefined Variable: Use set -u to catch typos in variable names:

    set -u
echo "Name: $nam"  # Error: nam is undefined (typo for name)

  • Check Exit Codes: Every command returns an exit code ($?). 0 = success, non-zero = failure:

    ls /nonexistent
if [ $? -ne 0 ]; then  # -ne = "not equal"
    echo "Command failed!"
fi

Signal Handling with trap

Use trap to run commands when the script receives signals (e.g., Ctrl+C or script exit). Common use cases: cleaning up temporary files.

Example: Clean up a temporary file on script exit:

#!/bin/bash
temp_file=$(mktemp)  # Create temp file
echo "Temporary file: $temp_file"

# Cleanup function
cleanup() {
    rm -f "$temp_file"
    echo "Cleaned up temporary file."
}

trap cleanup EXIT  # Run cleanup on script exit (normal or via Ctrl+C)

# Simulate work
sleep 10

4. Common Practices for Writing Shell Scripts

Script Organization and Documentation

Start scripts with a header comment describing their purpose, author, date, and usage:

#!/bin/bash
# Purpose: Backup /home directory to an external drive
# Author: Jane Doe
# Date: 2024-05-20
# Usage: ./backup.sh <external_drive_path>

Modular Design with Functions

Break large scripts into functions for reusability. For example, a backup script might have functions for:

  • check_drive_mount: Verify the external drive is mounted.
  • compress_files: Create a compressed archive.
  • log_message: Log activity to a file.

Commenting and Readability

  • Comment why, not what. Explain the purpose of complex logic, not obvious steps: 
    # Bad: "Loop through files" (obvious)
# Good: "Loop through logs to delete entries older than 30 days" (purpose)
  • Use consistent indentation (2 or 4 spaces) for loops/conditionals.

Version Control for Scripts

Store scripts in Git or another version control system to track changes, revert mistakes, and collaborate.

5. Best Practices for Robust Scripts

Security: Input Validation and Sanitization

  • Validate User Input: Never trust raw input. For example, if a script takes a filename as an argument, check if it exists:

    if [ ! -f "$1" ]; then
    echo "Error: $1 is not a valid file."
    exit 1
fi

  • Avoid Insecure Temporary Files: Use mktemp instead of hardcoding temporary file names (prevents race conditions):

    temp_file=$(mktemp)  # Secure: unique, random name

Performance Optimization

  • Use Built-in Commands: Avoid external commands (e.g., awk, sed) when Bash built-ins suffice (e.g., string manipulation with ${var%suffix}).
  • Minimize Subshells: Subshells (e.g., $(command)) are slow. Use process substitution or built-ins instead: 
    # Slow: Subshell for each line
while read line; do echo "$line"; done < file.txt

# Faster: Use built-in mapfile to load lines into an array
mapfile -t lines < file.txt
for line in "${lines[@]}"; do echo "$line"; done

Portability Across Shells

If your script might run on systems with non-Bash shells (e.g., sh), avoid Bash-specific features (e.g., arrays, [[ ]] conditionals). Use POSIX-compliant syntax:

# POSIX-compliant conditional (works in sh)
if [ "$name" = "Alice" ]; then ...  # Use [ ] instead of [[ ]]

Testing and Debugging Tools

  • ShellCheck: A static analysis tool to find bugs in shell scripts. Install via apt install shellcheck and run: 
    shellcheck my_script.sh
  • Debug Mode: Use bash -x my_script.sh to print each command as it runs (equivalent to set -x in the script).

6. Real-World Examples

Example 1: Automated Backup Script

This script backs up a directory to a compressed archive, checks disk space, and logs activity.

#!/bin/bash
set -euo pipefail  # Exit on error, undefined var, or pipe failure

# Configuration
SOURCE_DIR="/home/user/documents"
BACKUP_DIR="/mnt/backup"
LOG_FILE="$BACKUP_DIR/backup_log.txt"
DATE=$(date +%Y%m%d_%H%M%S)
ARCHIVE_NAME="backup_$DATE.tar.gz"

# Function to log messages
log() {
    echo "[$(date +%Y-%m-%d_%H:%M:%S)] $1" >> "$LOG_FILE"
}

# Check if source directory exists
if [ ! -d "$SOURCE_DIR" ]; then
    log "Error: Source directory $SOURCE_DIR not found."
    exit 1
fi

# Check free space on backup drive
FREE_SPACE=$(df -P "$BACKUP_DIR" | tail -1 | awk '{print $4}')  # Free blocks
REQUIRED_SPACE=$(du -s "$SOURCE_DIR" | awk '{print $1}')  # Source size in blocks
if [ "$FREE_SPACE" -lt "$REQUIRED_SPACE" ]; then
    log "Error: Insufficient space on backup drive."
    exit 1
fi

# Create backup
log "Starting backup of $SOURCE_DIR..."
tar -czf "$BACKUP_DIR/$ARCHIVE_NAME" -C "$SOURCE_DIR" .  # -C changes to source dir first

# Verify backup
if [ -f "$BACKUP_DIR/$ARCHIVE_NAME" ]; then
    log "Backup successful: $ARCHIVE_NAME"
else
    log "Error: Backup failed to create archive."
    exit 1
fi

Example 2: System Monitoring and Alerting

This script checks CPU and memory usage, sending an alert if thresholds are exceeded.

#!/bin/bash
set -euo pipefail

# Thresholds (percent)
CPU_THRESHOLD=80
MEM_THRESHOLD=80
ALERT_EMAIL="[email protected]"

# Get CPU usage (average over 1 minute)
CPU_USAGE=$(top -bn1 | grep "Cpu(s)" | awk '{print $2}' | cut -d. -f1)

# Get memory usage (used / total * 100)
MEM_USAGE=$(free | grep Mem | awk '{print $3/$2 * 100}' | cut -d. -f1)

# Check CPU
if [ "$CPU_USAGE" -gt "$CPU_THRESHOLD" ]; then
    echo "High CPU usage: $CPU_USAGE%" | mail -s "CPU Alert" "$ALERT_EMAIL"
fi

# Check memory
if [ "$MEM_USAGE" -gt "$MEM_THRESHOLD" ]; then
    echo "High memory usage: $MEM_USAGE%" | mail -s "Memory Alert" "$ALERT_EMAIL"
fi

7. Conclusion

Shell scripting is a cornerstone of Linux system administration and automation. By mastering the fundamentals (variables, loops, functions), adopting common practices (modular design, commenting), and following best practices (security, performance), you can write scripts that are robust, efficient, and easy to maintain.

The key to proficiency is practice: start with small scripts (e.g., automating file cleanup), gradually tackle more complex tasks (e.g., system monitoring), and use tools like ShellCheck to refine your code. With time, you’ll leverage shell scripting to save hours of manual work and gain deeper control over your Linux system.

8. References