Implementing QoS on a Linux Network: A Comprehensive Guide

In today’s networked world, where applications compete for limited bandwidth, ensuring critical traffic (e.g., VoIP, video conferencing, SSH) receives priority is essential. Quality of Service (QoS) is the set of techniques that enable this by managing network resources, reducing latency, minimizing jitter, and preventing packet loss for prioritized traffic. Linux, with its robust kernel and tools like tc (Traffic Control), netem, and iptables, offers powerful, open-source QoS capabilities. This blog will demystify Linux QoS, covering fundamental concepts, implementation steps, common practices, and best practices to help you optimize your network for reliability and performance.

Fundamentals of QoS on Linux
Implementing QoS: Step-by-Step
Common QoS Practices
Best Practices
Conclusion
References

1. Fundamentals of QoS on Linux

1.1 What is QoS?

QoS is a suite of techniques to prioritize network traffic, ensuring that critical applications (e.g., VoIP) meet performance requirements even when the network is congested. It answers questions like:

Which traffic should get priority?
How much bandwidth should each application/ user get?
How to handle congestion?

1.2 Key QoS Metrics

To implement QoS effectively, you must target these metrics:

Bandwidth: Maximum data rate (e.g., 100Mbps). QoS ensures critical traffic gets a guaranteed share.
Latency: Delay between sending and receiving a packet (critical for VoIP/ video).
Jitter: Variation in latency (disrupts real-time traffic like video calls).
Packet Loss: Percentage of packets dropped (ruins user experience for real-time apps).

1.3 Linux QoS Architecture

Linux QoS relies on the kernel’s traffic control subsystem, managed via the tc utility. At its core are three components:

Queuing Disciplines (qdiscs)

Qdiscs manage how packets are buffered, ordered, and transmitted on a network interface. Linux supports multiple qdiscs, each optimized for specific use cases:

pfifo_fast: Default qdisc; prioritizes packets by IP TOS (Type of Service) into 3 bands (high/medium/low).
HTB (Hierarchical Token Bucket): Flexible qdisc for partitioning bandwidth into hierarchical classes (e.g., “VoIP” → “Real-Time” → “Root”).
SFQ (Stochastic Fairness Queueing): Ensures fairness by distributing bandwidth evenly across flows (prevents one user from hogging the network).
RED (Random Early Detection): Congestion avoidance by dropping packets early to prevent buffer bloat.
netem: Emulates network conditions (latency, jitter, loss) for testing QoS.

Classes

Classes are sub-units of qdiscs that group traffic into categories (e.g., “VoIP”, “Web”, “Guest”). They enforce bandwidth limits and priorities.

Filters

Filters classify packets into classes based on criteria like port, IP address, or DSCP mark (e.g., “all port 5060 (SIP) traffic → VoIP class”).

2. Implementing QoS: Step-by-Step

2.1 Step 1: Define Requirements

Start by answering:

Which traffic is critical? (e.g., SSH, VoIP, video conferencing).
What are the bandwidth constraints? (e.g., 100Mbps WAN link, 10Mbps upload).
SLA targets? (e.g., VoIP latency < 150ms, jitter < 30ms).

2.2 Step 2: Traffic Classification

Classify traffic to identify what needs prioritization. Use these criteria:

Port: e.g., SSH (22), HTTP (80), VoIP (SIP: 5060, RTP: 10000-20000).
IP Address: e.g., management subnet (192.168.1.0/24), guest network (192.168.2.0/24).
DSCP Mark: End-to-end QoS uses DSCP (Differentiated Services Code Point) marks (e.g., EF for VoIP, AF41 for critical data).

2.3 Step 3: Choose a Queuing Discipline

Select a qdisc based on your goals:

HTB: Best for hierarchical bandwidth allocation (e.g., “10% to VoIP, 30% to Web”).
SFQ: Pair with HTB to ensure fairness within classes (prevents a single user from dominating “Web” class).
netem: Use to test QoS by emulating WAN conditions (e.g., “add 50ms latency + 1% loss” to simulate a slow link).

2.4 Step 4: Apply QoS with `tc` (Practical Examples)

Let’s implement QoS on a Linux router with an internet-facing interface eth0 (10Mbps upload). We’ll prioritize:

SSH (management): Port 22, 1Mbps guaranteed.
VoIP: SIP (5060) + RTP (10000-20000), 2Mbps guaranteed.
Web (HTTP/HTTPS): Ports 80/443, 5Mbps guaranteed.
Guests: Limit 192.168.2.0/24 to 1Mbps.

Example 1: Basic HTB Setup

First, clear existing rules and set up an HTB root qdisc on eth0 with a total bandwidth of 10Mbps:

# Clear existing QoS rules on eth0
tc qdisc del dev eth0 root 2>/dev/null

# Add HTB root qdisc with 10Mbps total bandwidth
tc qdisc add dev eth0 root handle 1: htb default 40  # Default class: 1:40 (other traffic)

Example 2: Create Classes

Define hierarchical classes under the root qdisc:

# Root class (10Mbps)
tc class add dev eth0 parent 1: classid 1:1 htb rate 10mbit ceil 10mbit

# Class 1:10 (SSH): 1Mbps, high priority (prio 1)
tc class add dev eth0 parent 1:1 classid 1:10 htb rate 1mbit ceil 1mbit prio 1

# Class 1:20 (VoIP): 2Mbps, priority 2
tc class add dev eth0 parent 1:1 classid 1:20 htb rate 2mbit ceil 2mbit prio 2

# Class 1:30 (Web): 5Mbps, priority 3
tc class add dev eth0 parent 1:1 classid 1:30 htb rate 5mbit ceil 5mbit prio 3

# Class 1:40 (Other traffic): Remaining 2Mbps, priority 4
tc class add dev eth0 parent 1:1 classid 1:40 htb rate 2mbit ceil 2mbit prio 4

# Class 1:50 (Guest network): 1Mbps, lowest priority (prio 5)
tc class add dev eth0 parent 1:1 classid 1:50 htb rate 1mbit ceil 1mbit prio 5

Example 3: Add Fairness with SFQ

Attach SFQ to each class to ensure fairness across flows (e.g., prevent one Web user from dominating the 5Mbps Web class):

for class in 10 20 30 40 50; do
  tc qdisc add dev eth0 parent 1:$class handle ${class}0: sfq perturb 10  # "perturb 10" rehashes flows every 10s
done

Example 4: Add Filters to Classify Traffic

Use filters to direct traffic into classes. Here, we match by port and IP:

# Filter 1: SSH (port 22) → class 1:10
tc filter add dev eth0 protocol ip parent 1: prio 1 u32 match ip dport 22 0xffff flowid 1:10

# Filter 2: VoIP (SIP port 5060 + RTP ports 10000-20000) → class 1:20
tc filter add dev eth0 protocol ip parent 1: prio 2 u32 match ip dport 5060 0xffff flowid 1:20
tc filter add dev eth0 protocol ip parent 1: prio 2 u32 match ip sport 10000 0xfff0 flowid 1:20  # RTP 10000-20000

# Filter 3: Web (HTTP/HTTPS ports 80/443) → class 1:30
tc filter add dev eth0 protocol ip parent 1: prio 3 u32 match ip dport 80 0xffff flowid 1:30
tc filter add dev eth0 protocol ip parent 1: prio 3 u32 match ip dport 443 0xffff flowid 1:30

# Filter 4: Guest network (192.168.2.0/24) → class 1:50
tc filter add dev eth0 protocol ip parent 1: prio 5 u32 match ip src 192.168.2.0/24 flowid 1:50

2.5 Step 5: Persist Configurations

tc rules are temporary (lost on reboot). To persist them:

Systemd Service: Create a service file (e.g., /etc/systemd/system/qos.service) to run tc commands on boot.
Network Scripts: Use /etc/network/interfaces (Debian) or /etc/sysconfig/network-scripts (RHEL) to apply rules post-interface-up.

3. Common QoS Practices

3.1 DSCP Marking for End-to-End QoS

For QoS across routers (e.g., home → ISP → office), use DSCP marks. Linux can mark packets with iptables, and tc can filter on these marks:

# Mark VoIP traffic with DSCP EF (46) using iptables
iptables -t mangle -A OUTPUT -p udp --dport 5060 -j DSCP --set-dscp 46

# Filter DSCP EF traffic into VoIP class with tc
tc filter add dev eth0 protocol ip parent 1: prio 2 u32 match ip dscp 46 0xff flowid 1:20

3.2 Policing vs. Shaping

Shaping: Delays excess traffic to stay within bandwidth limits (e.g., “shape guest traffic to 1Mbps”). Use htb with rate/ceil.

Policing: Drops excess traffic immediately (e.g., “police P2P traffic to 500Kbps”). Use tc police:

# Police P2P traffic (port 6881) to 500Kbps, drop excess
tc filter add dev eth0 protocol ip parent 1: prio 4 u32 match ip dport 6881 0xffff \
  police rate 500kbit burst 10k drop flowid 1:40

3.3 Inbound QoS with `ifb`

Linux QoS traditionally works on outbound traffic. For inbound traffic (e.g., prioritizing download of critical files), use the ifb (Intermediate Functional Block) driver to redirect inbound traffic to a virtual interface, then shape it:

# Load ifb module
modprobe ifb
ip link set dev ifb0 up

# Redirect inbound eth0 traffic to ifb0
tc qdisc add dev eth0 handle ffff: ingress
tc filter add dev eth0 parent ffff: protocol ip u32 match u32 0 0 action mirred egress redirect dev ifb0

# Shape ifb0 (inbound) like outbound traffic
tc qdisc add dev ifb0 root handle 1: htb default 40
# ... (add classes/filters to ifb0 as in outbound example)

4. Best Practices

4.1 Start with Baseline Measurements

Before implementing QoS, measure current network conditions (latency, jitter, throughput) using tools like iperf, mtr, or tcptrace. This helps validate QoS effectiveness later.

4.2 Keep Configurations Simple

Avoid over-engineering. Start with 3-4 classes (e.g., “Critical”, “Normal”, “Low”) and expand only if needed. Complexity leads to hard-to-debug issues.

4.3 Test Under Load

Simulate traffic with tools like:

iperf3 for bandwidth testing: iperf3 -c <server> -p 5201 -t 60
sipp for VoIP: sipp -sn uac <sip-server> -d 3600
netem for emulating WAN conditions: tc qdisc add dev eth0 root netem delay 50ms jitter 10ms loss 1%

4.4 Monitor and Iterate

Use these tools to validate QoS:

tc -s qdisc show dev eth0: Check packet counts, drops, and delays per qdisc/class.
iftop/nload: Monitor real-time bandwidth usage.
Wireshark: Verify DSCP marks (filter: ip.dsfield.dscp == 46 for VoIP EF).

4.5 Document and Version-Control

Store tc scripts in version control (Git) and document class/filter logic. Example script structure:

#!/bin/bash
# qos_setup.sh: QoS config for eth0 (10Mbps upload)
INTERFACE="eth0"
TOTAL_BW="10mbit"

# Clear old rules
tc qdisc del dev $INTERFACE root 2>/dev/null

# ... (rest of the HTB/filter commands)

5. Conclusion

Linux QoS, powered by tc, netem, and iptables, is a versatile tool for prioritizing critical traffic. By following the steps outlined—defining requirements, classifying traffic, choosing qdiscs, and applying filters—you can ensure reliable performance for essential applications.

Remember to start small, test rigorously, and monitor continuously. With practice, you’ll master Linux QoS to build a network that adapts to your needs.

Table of Contents