dotlinux guide

A Beginner’s Guide to Shell Scripts: From Basics to Advanced

Table of Contents

  1. What is a Shell Script?
  2. Basics of Shell Scripting
  3. Control Structures
  4. Functions
  5. Intermediate Concepts
  6. Advanced Concepts
  7. Common Practices
  8. Best Practices
  9. Conclusion
  10. References

What is a Shell Script?

A shell script is a text file containing a sequence of commands executed by a shell (e.g., Bash, Zsh, Sh). Shells are command-line interpreters that act as intermediaries between users and the operating system. Scripts automate tasks like file backups, log analysis, system monitoring, and more—eliminating the need to run commands manually.

Basics of Shell Scripting

Shebang Line

The first line of a shell script, called the “shebang,” specifies the shell to use for execution. For Bash (the most common shell), it’s:

#!/bin/bash

Without this line, the system uses the default shell (often sh, which has limited features). Always include the shebang to ensure consistency.

Variables

Variables store data for reuse. In Bash, declare variables without spaces around the = sign:

name="Alice"  # String variable
age=30        # Numeric variable

To access a variable, prefix it with $:

echo "Name: $name"  # Output: Name: Alice
echo "Age: $age"    # Output: Age: 30

Special Variables

Shells provide built-in variables for common tasks:

VariableDescription
$0Script name
$1, $2...Positional arguments (e.g., $1 = first input to the script)
$#Number of arguments
$@All arguments (as a list)
$?Exit status of the last command (0 = success, non-zero = error)
$$Process ID (PID) of the script

Running Scripts

To run a script:

  1. Save it with a .sh extension (convention, not required), e.g., my_script.sh.
  2. Make it executable with chmod +x my_script.sh.
  3. Execute it: ./my_script.sh (or bash my_script.sh to override the shebang).

Control Structures

Control structures let you add logic to scripts (e.g., “if X happens, do Y”).

Conditionals (if-else, case)

if-else Statements

Check conditions (e.g., file existence, numeric comparisons) with if:

#!/bin/bash

file="example.txt"

# Check if file exists
if [ -f "$file" ]; then  # `-f` = "is a regular file"
  echo "$file exists."
elif [ -d "$file" ]; then  # `-d` = "is a directory"
  echo "$file is a directory."
else
  echo "$file does NOT exist."
fi

Common condition flags:

  • -f file: File exists and is regular.
  • -d dir: Directory exists.
  • -z string: String is empty.
  • $a -eq $b: Numeric equality (e.g., 3 -eq 3).
  • $a > $b: String comparison (use [[ ]] for numeric >).

case Statements

Simplify multi-condition checks with case:

#!/bin/bash

echo "Enter a fruit (apple/orange/banana):"
read fruit

case "$fruit" in
  apple)
    echo "Apple is red."
    ;;
  orange)
    echo "Orange is orange."
    ;;
  banana)
    echo "Banana is yellow."
    ;;
  *)  # Default case (matches anything else)
    echo "Unknown fruit."
    ;;
esac

Loops (for, while, until)

for Loops

Iterate over lists (e.g., files, numbers):

#!/bin/bash

# Iterate over files in the current directory
for file in *; do
  echo "Found file: $file"
done

# Iterate over numbers (1 to 5)
for i in {1..5}; do
  echo "Count: $i"
done

while Loops

Run commands as long as a condition is true:

#!/bin/bash

count=1
while [ $count -le 5 ]; do  # Run until count > 5
  echo "Count: $count"
  count=$((count + 1))  # Increment count (arithmetic expansion)
done

until Loops

Run commands until a condition is true (opposite of while):

#!/bin/bash

count=5
until [ $count -lt 1 ]; do  # Run until count < 1
  echo "Countdown: $count"
  count=$((count - 1))
done

Functions

Functions group reusable code into named blocks. Define them with function name() { ... } or name() { ... }:

#!/bin/bash

# Function to greet a user
greet() {
  local name=$1  # `local` = variable only exists in the function
  echo "Hello, $name!"
}

# Call the function
greet "Bob"  # Output: Hello, Bob!

Return values: Use return for exit codes (0-255), or echo to return strings and capture with $(function).

Intermediate Concepts

Input/Output Redirection

Redirect command output to files or other commands:

  • >: Overwrite a file (e.g., echo "Hi" > file.txt).
  • >>: Append to a file (e.g., echo "More text" >> file.txt).
  • <: Read input from a file (e.g., grep "error" < log.txt).
  • |: Pipe output of one command to another (e.g., ls -l | grep ".sh").

Example: Save command output to a log file:

#!/bin/bash

# Run `date` and save output to "log.txt"
date > log.txt

# Append system info to "log.txt"
uname -a >> log.txt

echo "Log saved to log.txt"

Error Handling

Prevent scripts from failing silently with:

  • set -e: Exit immediately if any command fails (non-zero exit status).
  • set -u: Treat unset variables as errors.
  • set -o pipefail: Fail a pipe if any command in the pipe fails.

Add these at the top of scripts for robustness:

#!/bin/bash
set -euo pipefail  # Exit on error, unset var, or pipe failure

# If "missing_file.txt" doesn't exist, script exits here
grep "hello" missing_file.txt  # Fails, so script stops
echo "This line never runs."  # Unreachable

Arrays

Store multiple values in arrays:

#!/bin/bash

# Declare an array
fruits=("apple" "banana" "cherry")

# Access elements (0-based index)
echo "First fruit: ${fruits[0]}"  # Output: apple

# Iterate over array
for fruit in "${fruits[@]}"; do
  echo "Fruit: $fruit"
done

# Get array length
echo "Number of fruits: ${#fruits[@]}"  # Output: 3

Command Substitution

Capture output of a command into a variable with $(command) (preferred) or `command`:

#!/bin/bash

# Get current date (output of `date` stored in `current_date`)
current_date=$(date +"%Y-%m-%d")
echo "Today is $current_date"  # Output: Today is 2024-05-20

# Count .sh files in the current directory
sh_count=$(ls -l *.sh | wc -l)
echo "Number of .sh files: $sh_count"

Advanced Concepts

Debugging

Debug scripts with:

  • set -x: Print commands as they run (add at the top or run with bash -x script.sh).
  • trap: Catch signals (e.g., Ctrl+C) to clean up before exiting.

Example with set -x:

#!/bin/bash
set -x  # Enable debugging

name="Alice"
echo "Hello, $name"  # Script prints: + echo 'Hello, Alice' → Hello, Alice

Regular Expressions & Text Processing

Use tools like grep, sed, and awk with regex to parse text:

  • grep "pattern" file: Search for “pattern” in a file.
  • sed 's/old/new/g' file: Replace “old” with “new” globally in file.
  • awk '{print $1}' file: Print the first column of file.

Example: Find “error” lines in a log and count them:

#!/bin/bash
log_file="app.log"

# Count lines containing "error" (case-insensitive)
error_count=$(grep -ci "error" "$log_file")  # `-c` = count, `-i` = ignore case
echo "Total errors: $error_count"

File Manipulation

Automate file tasks like backups, renaming, or deletion with tools like find, cp, and mv:

#!/bin/bash
set -euo pipefail

# Backup .txt files to "backups/" directory
mkdir -p backups  # Create "backups" if it doesn't exist

for file in *.txt; do
  cp "$file" "backups/$file.bak"  # Copy file to backups with .bak extension
done

echo "Backup complete."

Environment Variables

Environment variables are global variables accessible to all processes. Use export to make variables available to child processes:

#!/bin/bash

# Set a local variable (only visible in this script)
local_var="I'm local"

# Export a variable (visible to subcommands)
export GLOBAL_VAR="I'm global"

# Subcommand inherits GLOBAL_VAR
bash -c 'echo "Subcommand sees: $GLOBAL_VAR"'  # Output: Subcommand sees: I'm global
echo "Script sees local: $local_var"  # Output: Script sees local: I'm local

Common Practices

  • Comment Liberally: Explain “why” (not just “what”) for readability.
  • Use Descriptive Names: backup_logs.sh instead of script1.sh.
  • Check Dependencies: Verify required commands exist (e.g., if ! command -v jq &> /dev/null; then echo "jq not installed"; exit 1; fi).
  • Handle Arguments: Validate inputs with if [ $# -eq 0 ]; then echo "Usage: $0 <file>"; exit 1; fi.

Best Practices

  • Quote Variables: Avoid word-splitting issues (e.g., echo "$name" instead of echo $name).
  • Use [[ ]] Over [ ]: For better syntax (e.g., [[ $age -gt 18 ]] instead of [ $age -gt 18 ]).
  • Lint with ShellCheck: Use ShellCheck to catch bugs (e.g., shellcheck my_script.sh).
  • Modularize with Functions: Break large scripts into reusable functions.
  • Test Incrementally: Test small parts of the script before combining them.

Conclusion

Shell scripting is a powerful skill for automating tasks and managing systems. Start with basics like variables and loops, then progress to error handling and text processing. Remember to write readable, robust scripts with tools like ShellCheck, and always test thoroughly.

Next steps: Explore advanced shells (Zsh), learn awk/sed deeply, or automate cloud workflows with scripts!

References