This content originally appeared on DEV Community and was authored by Mohammed yasir Khan
My Story:
As a fresher DevOps engineer, I didn't start with production experience - I started with curiosity. Most of my first steps in Linux came from setting up small projects on my laptop and cloud free-tiers: spinning up an EC2 instance, SSH-ing into it, breaking things, and then figuring out how to fix them. Every "command not found" error, every permission denied, taught me something new.
That's when it clicked for me: Linux isn't just an operating system, it's the foundation of DevOps and SRE. This guide is a collection of what I've learned so far - the exact concepts and commands that helped me go from "where do I even start?" to "I can confidently work on Linux in my DevOps journey."
This blog is a complete guide to Linux for DevOps & SRE engineers. We'll start with the basics - what Linux is, why it's used - and then dive deep into the most production-focused commands and real incident scenarios.
By the end, you'll have the practical knowledge to troubleshoot like a pro, and confidence that you're building skills that every top engineer relies on.
- What is Linux? Linux is an open-source operating system kernel first released by Linus Torvalds in 1991. Over the years, it grew into a family of distributions (distros) such as Ubuntu, CentOS, Debian, and Amazon Linux. Unlike Windows or macOS, Linux is: Free and open-source - anyone can inspect, modify, and distribute it. Stable and secure - runs most of the world's servers and cloud infrastructure. Lightweight and customizable - you only install what you need, no bloat.
💡 Fun fact: Over 90% of cloud servers and 100% of Kubernetes clusters run on Linux. If you're in DevOps/SRE, Linux isn't optional - it's the default.
- Why Linux is Used in DevOps & SRE So, why does every DevOps/SRE engineer swear by Linux? Runs the Cloud: All major cloud providers (AWS, GCP, Azure) use Linux-based images for VMs, containers, and managed services. Automation-Friendly: Tools like Ansible, Terraform, and Kubernetes integrate naturally with Linux. Security: Strong permission model and open-source review make it a security-first OS. Flexibility: From a Raspberry Pi to a 1000-node Kubernetes cluster, Linux scales everywhere. Ecosystem: Docker, Kubernetes, Jenkins, Prometheus - all cloud-native tooling runs best on Linux.
👉 Example: When your AWS EC2 server is unresponsive, you'll likely SSH into a Linux box, check logs in /var/log/, restart services with systemctl, and debug with journalctl.
That's why Linux mastery = faster incident resolution.
- Problems Linux Solves What? Before Linux, organizations relied on proprietary, closed-source operating systems (like Windows Server or Solaris) that were costly, less flexible, and not cloud-native. Why? Linux solves key problems that modern DevOps/SRE teams face: Vendor Lock-in: Proprietary OS meant paying high license fees. Linux is open-source and free. Scalability Issues: Old systems weren't built for cloud-native workloads; Linux scales effortlessly. Limited Automation: Windows had GUI-heavy operations; Linux enables automation with scripts and tools. Performance & Stability: Linux is lightweight, stable, and can run for months without a reboot. Community & Ecosystem: Thousands of engineers contribute, so bugs/security issues are fixed quickly.
👉 Example: If your Kubernetes cluster runs 500 microservices, Linux ensures consistency across all nodes. Running that on a GUI-heavy OS would be inefficient, costly, and unstable.
- Advantages & Disadvantages of Linux Advantages (Why Linux is Loved in DevOps/SRE) Open Source & Free: No license fees, huge community support. Security: Strong permission and user model, SELinux, AppArmor. Stability & Reliability: Servers can run for years without downtime. Performance: Optimized for resource usage - perfect for cloud workloads. Customizability: **Choose minimal distros for containers or full-featured ones for enterprise. **Automation-First: Command-line interface is powerful and scriptable. **Cloud-Native: **All major cloud tools and orchestration platforms are Linux-first.
Disadvantages (Where Linux is Hard)
Steep Learning Curve: For beginners, CLI feels overwhelming.
Fragmentation: Many distros, slightly different commands (apt vs yum).
Desktop Usage: Limited adoption in consumer laptops → less exposure before entering industry.
Support Needs: Enterprise support often requires Red Hat/SUSE subscriptions.
👉 Tip for Juniors: Don't worry about mastering all distros. Focus on 1–2 (like Ubuntu + CentOS/RHEL) and build confidence.
- Linux Distributions (Distros) for DevOps/SRE What? A Linux distribution (distro) is a version of Linux that includes the kernel + package managers + tools. Each distro is tailored for different use cases. Why? Different distros are optimized for enterprise servers, cloud workloads, containers, or desktop usage. As a DevOps/SRE, you'll encounter multiple distros, so you should know where each fits. 🔹 Common Distros & Their Use Cases figure: Different Types of Linus Destros👉 Example in Real Work: If you spin up an AWS EC2 instance, you'll often see Amazon Linux as default. If you work in an enterprise with strict compliance, they'll likely use RHEL. If you're practicing as a beginner, Ubuntu is the most friendly starting point.
📚 Files & Directories in Linux
What is a Filesystem?
A filesystem is how an operating system organizes and stores data on disk. In Linux, everything is a file - regular files, directories, devices, sockets, even processes.
👉 Example:/dev/sda (your disk) is treated like a file, /proc/cpuinfo (your CPU info) is also a file.
/ → Root directory (everything starts here)
├── bin/ → Essential user binaries
│ ├── ls # list files
│ ├── cp # copy files
│ └── mv # move/rename files
├── boot/ → Bootloader & kernel files
│ ├── vmlinuz # Linux kernel image
│ ├── initrd.img # initial RAM disk for boot
│ └── grub/ # GRUB bootloader configs
├── dev/ → Device files (hardware abstraction)
│ ├── sda # disk drives
│ ├── tty # terminals
│ └── null # null device
├── etc/ → System configuration files
│ ├── passwd # user accounts
│ ├── shadow # encrypted passwords
│ ├── group # group definitions
│ ├── ssh/ # SSH server configs
│ └── nginx/ # web server configs
├── home/ → Home directories for regular users
│ ├── alice/ # user alice's data
│ └── bob/ # user bob's data
├── lib/ → Shared libraries
│ ├── libpthread.so.0 # threading library
│ └── libm.so # math library
├── media/ → Mount points for removable media
│ └── usb/ # USB mount point
├── mnt/ → Temporary mount points
│ └── iso/ # mounted ISO images
├── opt/ → Optional/third-party software
│ └── docker/ # docker installation files
├── proc/ → Virtual filesystem for kernel & processes
│ ├── cpuinfo # CPU details
│ ├── meminfo # Memory details
│ └── 1234/ # process info (PID 1234)
├── root/ → Home directory for root user
├── sbin/ → System binaries (admin tasks)
│ ├── shutdown # shutdown server
│ ├── mount # mount/unmount filesystems
│ └── ifconfig # network configuration (older)
├── tmp/ → Temporary files (cleared on reboot)
├── usr/ → User programs & libraries
│ ├── bin/ # non-essential binaries
│ ├── lib/ # libraries
│ ├── include/ # header files for compilation
│ └── share/ # shared resources (docs, man pages)
└── var/ → Variable data
├── log/ # system logs
│ ├── syslog # general logs
│ └── auth.log # authentication logs
├── spool/ # mail & print queues
└── cache/ # cached data for apps
Explanation & Best Practices for Each Directory
/bin – Essential user commands
Use: Commands every user needs, even in single-user mode.
Best practice: Don't add custom binaries here; use /usr/local/bin/ instead.
/boot – Boot files
Use: Kernel images, bootloader configs.
Best practice: Only modify with caution. Backup configs before upgrading kernels.
/dev – Devices
Use: Represents hardware devices as files. For example, /dev/sda = first disk.
Best practice: Never edit these files directly; use tools like mount or fdisk.
/etc – Configurations
Use: Central place for system & application configs.
Best practice: Backup before editing; use version control for configs if possible (/etc Git repo).
/home – User data
Use: Personal files, scripts, and workspace for users.
Best practice: Limit sensitive data in home dirs; backup regularly.
/lib – Libraries
Use: Required by binaries in /bin and /sbin.
Best practice: Don't manually delete libraries; use package managers.
/media & /mnt – Mount points
Use: Temporary or removable storage.
Best practice: Unmount devices after use to avoid data corruption.
/opt – Optional software
Use: Third-party software installations.
Best practice: Install large apps here (Docker, Oracle, custom software) to avoid cluttering /usr/bin.
/proc – Virtual filesystem
Use: Access kernel and process info.
Best practice: Read-only; don't try to write files here (except in specific tuning like /proc/sys).
/root – Root home
Use: Admin user workspace.
Best practice: Don't store scripts for multiple users here.
/sbin – System binaries
Use: Commands for system administration.
Best practice: Use sudo to run commands from here; don't add personal scripts.
/tmp – Temporary files
Use: Scratch files, temporary data for apps.
Best practice: Avoid storing important data; auto-cleaned on reboot.
/usr – User programs & resources
Use: Non-essential binaries, libraries, documentation.
Best practice: Don't modify manually; use package managers.
/var – Variable data
Use: Logs, caches, print/mail spools.
Best practice: Rotate logs regularly (logrotate) and monitor disk usage (du/df).
Linux File & Directory Commands - DevOps/SRE Cheat Sheet
- Create, Update, Delete Files touch file.txt # Create an empty file echo "Hello World" > file.txt # Create file with content (overwrite if exists) echo "Append this line" >> file.txt # Append content to file cat > newfile.txt # Create file and type content (Ctrl+D to save) rm file.txt # Delete a file rm -i file.txt # Delete with prompt
- Directories - Create, Delete, Navigate mkdir mydir # Create directory mkdir -p parent/child/grand # Create nested directories rmdir emptydir # Remove empty directory rm -rf mydir # Remove non-empty directory recursively pwd # Print current directory cd /path/to/dir # Change directory cd ~ # Go to home directory cd - # Go to previous directory ls # List files in current directory ls -l # List with detailed info (permissions, owner, size) ls -a # List including hidden files
- Copy, Move, Rename cp file.txt /backup/ # Copy file to another location cp -r mydir /backup/ # Copy directory recursively mv file.txt newfile.txt # Rename a file mv file.txt /new/path/ # Move file to new location
- Permissions & Metadata ls -l file.txt # Show permissions, owner, size, modification stat file.txt # Detailed metadata of file chmod 644 file.txt # Set permissions (owner=r/w, group/others=r) chmod +x script.sh # Make file executable chown user:group file.txt # Change owner and group umask # Show default permission mask
- Disk Usage & Space df -h # Show disk usage per filesystem (human-readable) du -sh * # Show size of files/directories in current folder du -sh /var/log/* # Find size of logs du -ah /home/ubuntu/ | sort -rh | head -10 # Top 10 largest files/directories
- Search Files & Content find /var/log -name "*.log" # Find all .log files find /home -type f -size +100M # Find files larger than 100MB grep "error" file.txt # Search for "error" inside file grep -i "timeout" /etc/nginx/ -r # Recursive search, case-insensitive grep "failed" /var/log/auth.log | awk '{print $1, $2, $5}' # Extract date & user
- Archiving & Syncing tar -czvf archive.tar.gz /path/to/dir # Compress directory into tar.gz tar -xzvf archive.tar.gz -C /restore/ # Extract archive to target folder rsync -av /source/ /destination/ # Sync directories recursively (preserve attributes) rsync -av --dry-run /source/ /destination/ # Test sync without changes
Linux Processes & Services
Processes:
A process is any running program in Linux. Each command you execute, every service, script, or daemon is a process.
Why it matters: Processes consume CPU, memory, and other resources. Monitoring them helps prevent crashes, high CPU usage, and memory leaks.
Services (Daemons):
A service is a long-running background process that provides a function, like a web server (nginx), database (mysql), or monitoring agent (prometheus).
Why it matters: Services ensure your applications, monitoring, and infrastructure run continuously.
Key Concept:
Every service is a process, but not every process is a service. Services are usually managed with systemd/systemctl for startup, restart, and monitoring.
Linux Processes & Services Commands - DevOps/SRE Cheat Sheet
- Process Monitoring with ps ps # List processes for current shell ps -ef # Full list of all processes ps aux # Alternative format showing user, CPU, memory ps -u ubuntu # Show processes by user 'ubuntu' Example: When a service fails, check if it's running:
ps -ef | grep nginx
Helps identify if the process crashed or didn't start.
Real-time Process Monitoring with top / htop
top # Interactive real-time process viewer
htop # Enhanced interactive view (if installed)
top -u ubuntu # Show processes for specific user
Example:
CPU spike alert: top shows which process is consuming CPU → investigate memory leaks or runaway processes.Managing Process Priority (nice / renice)
nice -n 10 ./backup.sh # Start process with lower priority (10)
renice -n 5 -p 1234 # Change priority of PID 1234 to 5
Example:
During high-load on production, lower priority of non-critical scripts so main services remain responsive.Tracing Process Calls (strace / ltrace)
strace -p 1234 # Trace system calls of PID 1234
strace -o output.log -p 1234 # Save trace output to file
ltrace ./app # Trace library calls of a binary
Example:
Identify why a program fails to open a file → strace shows "Permission denied" or missing library.Service Management (systemctl / service)
systemctl status nginx # Check status of nginx service
systemctl start nginx # Start service
systemctl stop nginx # Stop service
systemctl restart nginx # Restart service
systemctl enable nginx # Enable service at boot
systemctl disable nginx # Disable service at boot
systemctl list-units --type=service # List all active services
Example:
After deployment, nginx not serving pages:
systemctl status nginx
journalctl -u nginx -b
Shows logs and reasons why it failed to start.
- Log Analysis with journalctl journalctl # View all system logs journalctl -u nginx # Logs for specific service journalctl -f # Follow logs in real-time (like tail -f) journalctl --since "2 hours ago" # Logs from last 2 hours journalctl -p err # Only show error-level logs Example: Troubleshooting system boot failures →
journalctl -b -p err
Displays errors only from the last boot.
Best Practices for Processes & Services
Always check logs first with journalctl before restarting services.
Use systemctl enable for critical services to auto-start after reboot.
Monitor resource usage (top, htop) to detect runaway processes early.
Use strace/ltrace for deep debugging only when necessary - can be CPU intensive.
Prioritize scripts with nice to avoid disrupting production workloads.
Linux Networking
- What is Networking in Linux? Networking is how computers and devices communicate with each other over a network (LAN, WAN, or the Internet). Linux networking involves interfaces, IP addresses, ports, protocols, and routing.
Why it matters for DevOps/SRE:
Troubleshooting connectivity issues
Monitoring network traffic and latency
Ensuring services are reachable and secure
IP Address and Interface Management (ip)
ip addr show # Show all network interfaces and IP addresses
ip link show # Show interface details (up/down, MTU)
ip route show # Show routing table
ip addr add 192.168.1.10/24 dev eth0 # Assign IP to interface
ip link set eth0 up # Bring interface up
ip link set eth0 down # Bring interface down
Example:
ip addr show
Output shows your server's network interfaces and assigned IPs.Socket Statistics (ss)
ss -tuln # Show listening TCP/UDP ports with numeric info
ss -s # Summary of socket connections
ss -p # Show process using each socket
Example:
ss -tuln
Shows which ports are open and which services are listening.Legacy Network Connections (netstat)
netstat -tuln # Show active TCP/UDP ports (similar to ss)
netstat -rn # Show routing table
netstat -p # Show PID of processes using ports
Example:
netstat -tuln
Lists active ports and listening services.Network Requests (curl)
curl http://example.com # Fetch HTTP response from URL
curl -I http://example.com # Show only HTTP headers
curl -v http://example.com # Verbose output with connection info
Example:
curl -I https://google.com
Checks HTTP headers from Google to verify connectivity and response.DNS Queries (dig / nslookup)
dig example.com # Get DNS records for domain
dig A example.com # Get A record (IP)
dig MX example.com # Get mail server records
nslookup example.com # Alternative to dig
Example:
dig google.com
Shows IP addresses and DNS info for Google.Network Path Tracing (traceroute)
traceroute example.com # Trace path packets take to destination
traceroute -n example.com # Numeric IP only, skip DNS resolution
Example:
traceroute github.com
Shows each hop from your server to GitHub, useful for latency analysis.Packet Capture (tcpdump)
tcpdump -i eth0 # Capture all packets on interface eth0
tcpdump -i eth0 port 80 # Capture HTTP traffic
tcpdump -i eth0 -nn # Numeric addresses, no DNS resolution
tcpdump -r capture.pcap # Read previously saved capture file
Example:
tcpdump -i eth0 port 22
Monitors SSH traffic for troubleshooting connection issues.
Linux Users, Groups & Permissions
Users:
A user is anyone or any system entity that can log in to the Linux system. Each user has a username, password, home directory, and a set of permissions that define what they can access.
Example: alice, root, ubuntu
Groups:
A group is a collection of users. Permissions can be assigned to groups to simplify access control.
Example: sudo, devops, docker
Permissions:
Linux uses read (r), write (w), execute (x) permissions for files and directories, assigned to owner, group, and others.
Why it matters for DevOps/SRE:
Control access to sensitive files (/etc, /var/log)
Ensure only authorized users can run scripts or manage services
Enforce security best practices via least privilege
Linux Users, Groups & Permissions - DevOps/SRE Guide
User Management (useradd, usermod, userdel)
sudo useradd alice # Create a new user 'alice'
sudo adduser alice # Create a new user 'alice'
sudo passwd alice # Set password for 'alice'
sudo usermod -aG sudo alice # Add 'alice' to 'sudo' group
sudo userdel -r olduser # Delete user 'olduser' and remove home directory
Example:
sudo useradd john
sudo passwd john
sudo usermod -aG docker john
Creates user john, sets a password, and adds him to the docker group for container management.Group Management (groupadd, gpasswd, groups)
sudo groupadd devops # Create a new group 'devops'
sudo gpasswd -a alice devops # Add user 'alice' to group 'devops'
groups alice # Show groups 'alice' belongs to
sudo gpasswd -d alice devops # Remove 'alice' from group 'devops'
Example:
sudo groupadd monitoring
sudo gpasswd -a john monitoring
groups john
Manages access for multiple users to shared resources (logs, scripts, etc.).File & Directory Permissions (chmod, chown, chgrp)
ls -l file.txt # View permissions, owner, group
chmod 644 file.txt # rw-r--r-- → owner=read/write, others=read
chmod +x script.sh # Make script executable
chown alice:devops file.txt # Change owner to 'alice' and group to 'devops'
chgrp devops file.txt # Change only the group
umask 002 # Default permission mask for newly created files
Example:
chmod 755 /home/alice/scripts/deploy.sh
chown alice:devops /var/log/myapp.log
Ensures only authorized users can execute scripts or read/write logs.Password Management (passwd, chage)
sudo passwd alice # Change password for 'alice'
sudo passwd -l alice # Lock user account (disable login)
sudo passwd -u alice # Unlock user account
sudo chage -l alice # View password expiry info
sudo chage -M 90 alice # Set password to expire every 90 days
Example:
sudo passwd -l john # Temporarily lock a user during offboarding
sudo chage -M 60 john # Force password rotation every 60 days
Helps maintain security best practices.Sudo Access & Safety (sudoers)
sudo visudo # Edit sudoers file safely
alice ALL=(ALL) NOPASSWD:ALL # Give 'alice' passwordless sudo
Best Practice:
Always use visudo to prevent syntax errors.
Limit sudo access to essential users only.
Best Practices for Users & Permissions
Principle of Least Privilege: Only give users the access they need.
Groups for Shared Access: Use groups instead of giving sudo/root access.
Secure Sensitive Files: Limit access to logs, configs, and scripts.
Use umask and chmod wisely: Ensure new files have proper permissions.
Audit regularly: Check /etc/passwd, /etc/group, /etc/sudoers for anomalies.
Logs in Linux
Logs are records of events, activities, or messages generated by the system, applications, and services. They help in monitoring, debugging, auditing, and troubleshooting issues on Linux servers.
Logs are essential for DevOps/SRE to detect incidents, understand system behavior, and perform Root Cause Analysis (RCA).
Linux stores logs in plain text files, making it easy to read, search, and process.
- Systemd Logs - journalctl journalctl is used to view logs managed by systemd (modern Linux systems). Example:
journalctl -u nginx --since "1 hour ago"
Shows logs for nginx service from the last hour.
Useful for real-time debugging or checking recent service failures.
Other useful options:
journalctl -f # Follow logs in real-time
journalctl -b # Logs since last boot
journalctl -p err # Show only error-level logs
Traditional Log Files (/var/log)
Linux also stores logs in the /var/log directory. Common logs include:
Example:
less /var/log/syslog
tail -f /var/log/nginx/access.log
less → scroll logs
tail -f → monitor logs in real-timeLog Rotation - logrotate
Problem: Logs grow over time and can fill up disk space.
Solution: logrotate automatically rotates, compresses, and deletes old logs.
Example /etc/logrotate.d/nginx config snippet:
/var/log/nginx/*.log {
daily
rotate 14
compress
missingok
notifempty
create 0640 www-data adm
sharedscripts
postrotate
systemctl reload nginx
endscript
}
Rotates nginx logs daily, keeps 14 old copies, compresses them, and reloads service after rotation.
- Searching & Filtering Logs Use grep pipelines to find relevant entries quickly. Examples:
grep -i error /var/log/app.log | less # Search "error" in logs
grep -i timeout /var/log/nginx/error.log # Search "timeout" in nginx error log
tail -f /var/log/syslog | grep CRON # Real-time monitoring for cron jobs
-i → case-insensitive
| less → scrollable output
Best Practices for Logs
Centralize logs using tools like ELK stack (Elasticsearch, Logstash, Kibana) or Prometheus + Grafana.
Always rotate and compress logs to save disk space.
Use grep, awk, and pipelines to filter and analyze logs efficiently.
Monitor critical logs in real-time for proactive incident detection.
Package Management & Updates
What is Package Management?
Package management is the process of installing, updating, removing, and maintaining software on a Linux system.
Linux uses package managers to automate software handling instead of manually downloading binaries.
Why it matters for DevOps/SRE:
Keep servers up-to-date and secure
Install essential tools and dependencies consistently
Automate patching and deployments
Debian/Ubuntu - apt
sudo apt update # Update package index (repo info)
sudo apt upgrade # Upgrade all installed packages
sudo apt install nginx # Install a package (nginx web server)
sudo apt remove nginx # Remove package but keep config files
sudo apt purge nginx # Remove package + config files
sudo apt search docker # Search for a package
sudo apt show nginx # Show package details
Example:
sudo apt update
sudo apt install htop
Updates repo info and installs htop for process monitoring.RHEL/CentOS/Fedora - yum / dnf
yum → traditional package manager
dnf → newer, faster replacement in Fedora/RHEL 8+
Commands (YUM):
sudo yum update # Update all packages
sudo yum install httpd # Install Apache web server
sudo yum remove httpd # Remove Apache
sudo yum search nginx # Search for a package
sudo yum info nginx # Show package info
Commands (DNF):
sudo dnf update # Update all packages
sudo dnf install nginx # Install nginx
sudo dnf remove nginx # Remove nginx
sudo dnf list installed # List all installed packages
sudo dnf repoquery nginx # Show package info from repo
Example:
sudo dnf install git
sudo dnf remove nano
Installs Git and removes nano editor.
- Unattended / Automatic Updates Keeps system patched automatically without manual intervention. Important for security patches in production servers.
Ubuntu/Debian:
sudo apt install unattended-upgrades
sudo dpkg-reconfigure --priority=low unattended-upgrades
Configures automatic security updates.
RHEL/CentOS/Fedora:
sudo yum install yum-cron
sudo systemctl enable --now yum-cron
Enables automatic updates in the background.
Best Practices for Package Management
Always update package index before installing
sudo apt update sudo dnf check-update
- Prefer security updates first on production servers.
- Use automation for multiple servers (Ansible, Puppet, or scripts).
- Keep a record of installed packages for auditing: dpkg --get-selections > packages_list.txt # Debian/Ubuntu rpm -qa > packages_list.txt # RHEL/CentOS
- Test updates in staging before production to avoid downtime.
With this section, Linux fundamentals guide is complete covering:
Linux Overview & File System
File & Directory Commands
Processes & Services
Networking Commands
Users, Groups & Permissions
Logs
Package Management & Updates
This content originally appeared on DEV Community and was authored by Mohammed yasir Khan
Mohammed yasir Khan | Sciencx (2025-09-25T08:47:50+00:00) Linux Fundamentals for DevOps & SRE: The Only Guide You’ll Ever Need. Retrieved from https://www.scien.cx/2025/09/25/linux-fundamentals-for-devops-sre-the-only-guide-youll-ever-need/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.