In the world of Linux system administration, DevOps, and everyday computing, repetitive tasks can drain time and introduce human error. Whether it’s backing up files, cleaning logs, monitoring system resources, or deploying applications, automation is the key to efficiency. Linux command line scripts—powered by shells like Bash—offer a lightweight, flexible way to automate these tasks without the need for complex programming languages. This blog will guide you through the fundamentals of Linux command line scripting for automation, from basic syntax to advanced best practices. By the end, you’ll be equipped to write robust, maintainable scripts to streamline your workflow and reduce manual effort.
Table of Contents
- Introduction
- Fundamental Concepts of Linux Shell Scripting
- Usage Methods: Creating and Running Scripts
- Common Automation Tasks with Script Examples
- Best Practices for Reliable Scripts
- Conclusion
- References
Fundamental Concepts of Linux Shell Scripting
What is a Shell Script?
A shell script is a text file containing a sequence of Linux commands and control structures (loops, conditionals) that the shell (e.g., Bash, Zsh) executes sequentially. Scripts automate repetitive tasks by bundling commands into a reusable program.
Shebang Line
The first line of a script, called the “shebang,” specifies the interpreter to use. For Bash scripts, this is:
#!/bin/bash Or, for portability (uses the system’s bash path):
#!/usr/bin/env bash Basic Syntax
Variables
Store data for reuse. Declare without spaces; access with $:
#!/usr/bin/env bash
NAME="John"
echo "Hello, $NAME!" # Output: Hello, John! Loops
Automate repetitive actions.
For Loop (iterate over items):
#!/usr/bin/env bash
# List all .txt files in the current directory
for file in *.txt; do
echo "Processing: $file"
done While Loop (run until a condition fails):
#!/usr/bin/env bash
COUNT=1
while [ $COUNT -le 5 ]; do
echo "Count: $COUNT"
COUNT=$((COUNT + 1)) # Increment count
done Conditionals
Execute commands based on conditions (e.g., file existence, exit codes).
If-Else Statement:
#!/usr/bin/env bash
FILE="data.txt"
if [ -f "$FILE" ]; then # Check if file exists
echo "$FILE exists."
else
echo "$FILE not found."
fi Common condition checks:
-f $FILE: File exists and is a regular file-d $DIR: Directory exists-z $VAR: Variable is empty$? -eq 0: Last command succeeded
Command Substitution
Capture output of a command into a variable using $(command) or backticks `command`:
#!/usr/bin/env bash
CURRENT_DATE=$(date +%Y-%m-%d) # Store today's date
echo "Today is $CURRENT_DATE" # Output: Today is 2024-05-20 Usage Methods: Creating and Running Scripts
Step 1: Create a Script File
Use a text editor like nano, vim, or VS Code to create a .sh file:
nano my_script.sh Step 2: Make It Executable
Grant execute permissions with chmod:
chmod +x my_script.sh Step 3: Run the Script
Execute the script directly (ensure it’s in your PATH or use a relative/absolute path):
./my_script.sh # Relative path (current directory)
/home/user/scripts/my_script.sh # Absolute path Scheduling Scripts with Cron
To automate scripts at specific times, use cron (a time-based job scheduler).
Cron Syntax
Cron jobs are defined in a crontab file with the format:
* * * * * command_to_run
- - - - -
| | | | |
| | | | +-- Day of the week (0-6, 0=Sunday)
| | | +---- Month (1-12)
| | +------ Day of the month (1-31)
| +-------- Hour (0-23)
+---------- Minute (0-59) Common Examples
- Run daily at 2 AM:
0 2 * * * /home/user/scripts/backup.sh - Run weekly on Sunday at 3 PM:
0 15 * * 0 /home/user/scripts/clean_logs.sh
Edit Crontab
Open the crontab editor for the current user:
crontab -e Add your job (e.g., daily backup) and save. Verify with:
crontab -l # List current cron jobs Common Automation Tasks with Script Examples
1. Automated Backups
Use rsync to back up files to an external drive or remote server.
Script: backup.sh
#!/usr/bin/env bash
# Backup Documents to external drive with logging
SOURCE="/home/user/Documents"
DEST="/mnt/external_drive/backups"
LOG_FILE="/var/log/backup.log"
DATE=$(date +%Y-%m-%d_%H-%M-%S)
# Create backup directory if it doesn't exist
mkdir -p "$DEST/$DATE"
# Run rsync and log output
rsync -av --delete "$SOURCE/" "$DEST/$DATE/" >> "$LOG_FILE" 2>&1
# Check if rsync succeeded
if [ $? -eq 0 ]; then
echo "[$DATE] Backup completed successfully" >> "$LOG_FILE"
else
echo "[$DATE] Backup FAILED" >> "$LOG_FILE"
exit 1
fi Schedule with Cron: Run daily at 2 AM:
0 2 * * * /home/user/scripts/backup.sh 2. Log Rotation/Cleanup
Delete old log files to free disk space.
Script: clean_logs.sh
#!/usr/bin/env bash
# Delete log files older than 30 days in /var/log/app
LOG_DIR="/var/log/app"
DAYS=30
# Delete files older than 30 days
find "$LOG_DIR" -name "*.log" -type f -mtime +$DAYS -delete
echo "[$(date)] Deleted logs older than $DAYS days in $LOG_DIR" >> /var/log/clean_logs.log 3. System Monitoring (Disk Usage Alert)
Send an email alert if disk usage exceeds a threshold (e.g., 90%).
Script: disk_alert.sh
#!/usr/bin/env bash
# Alert if disk usage exceeds 90%
THRESHOLD=90
EMAIL="[email protected]"
DISK_USAGE=$(df -h / | awk 'NR==2 {print $5}' | sed 's/%//')
if [ "$DISK_USAGE" -gt "$THRESHOLD" ]; then
echo "Disk usage is critical: $DISK_USAGE% on $(hostname)" | mail -s "Disk Alert: $(hostname)" "$EMAIL"
fi Note: Install mailutils for email support: sudo apt install mailutils.
Best Practices for Reliable Scripts
Use a Consistent Shebang
Always start with #!/usr/bin/env bash (portable) instead of #!/bin/bash (hardcoded path).
Add Comments for Clarity
Explain why not just what the code does:
#!/usr/bin/env bash
# Purpose: Backup user Documents to external drive
# Frequency: Daily via cron
# Dependencies: rsync Handle Errors Gracefully
Use set -euo pipefail to exit on errors, unset variables, or failed pipes:
#!/usr/bin/env bash
set -euo pipefail # Exit on error, unset var, or pipe failure
# Script will exit if $UNDEFINED_VAR is used (unset)
echo "This will fail: $UNDEFINED_VAR" Validate Inputs
Check for required arguments or dependencies:
#!/usr/bin/env bash
if [ $# -eq 0 ]; then
echo "Error: No input file provided."
echo "Usage: $0 <input_file>"
exit 1
fi
INPUT_FILE="$1"
if [ ! -f "$INPUT_FILE" ]; then
echo "Error: File $INPUT_FILE not found."
exit 1
fi Use Absolute Paths
Avoid relative paths (depend on working directory). Use absolute paths:
# Bad: cd Documents; rsync ... (fails if run from another directory)
# Good:
SOURCE="/home/user/Documents"
rsync -av "$SOURCE" ... Log Script Activity
Log to a file instead of relying on stdout:
LOG_FILE="/var/log/script.log"
echo "[$(date)] Script started" >> "$LOG_FILE"
# ... commands ...
echo "[$(date)] Script finished" >> "$LOG_FILE" Test Thoroughly
- Run with
bash -x script.shto debug (prints each command). - Use
shellcheck(a linter) to catch errors:sudo apt install shellcheck shellcheck my_script.sh
Avoid Hardcoded Credentials
Store secrets in environment variables or encrypted files:
# Bad: PASSWORD="secret123"
# Good:
PASSWORD="$DB_PASSWORD" # Load from environment Ensure Idempotency
Scripts should run safely multiple times (e.g., check if a file exists before creating it):
# Create directory only if it doesn't exist
mkdir -p "/path/to/dir" # -p avoids error if dir exists Conclusion
Linux command line scripts are a powerful tool for automating repetitive tasks, from backups to system monitoring. By mastering basic syntax, leveraging cron for scheduling, and following best practices like error handling and logging, you can build reliable, maintainable automation workflows. Start small—automate one task, then expand. With practice, you’ll transform hours of manual work into a few lines of code.