This content originally appeared on Level Up Coding - Medium and was authored by Mohammed Shamim
AppArmor for docker and kubernetes
AppArmor is a Linux kernel security module that allows the system administrator to restrict programs’ capabilities with per-program profiles. Profiles can allow capabilities like network access, raw socket access, and the permission to read, write, or execute files on matching paths.
In this article, we will discuss how we can use AppArmor Profiles along with docker containers and kubernetes pods.
If you are new to the AppArmor concepts such as how to use AppArmor and how to create an AppArmor profile, then I would highly suggest you take a look at some of the below resources :
The Comprehensive Guide To AppArmor
Quick Profile Language
AppArmor For Docker
By default, the docker container will run with docker-default AppArmor Profile unless we override it with the security-opt option.
When we install docker runtime into the system it automatically generates an AppArmor profile. If we execute the aa-status on the system, we will see that docker-default is already loaded.
>> aa-status
----------------------------------------------------------------------------
apparmor module is loaded.
30 profiles are loaded.
30 profiles are in enforce mode.
....
docker-default # <--***
....
Without Custom AppArmor Profile
Now, let’s create a nginx container named nginx-container :
>> docker run -p 80:80 -d --name nginx-container nginx
Now, execinto the pod and try to use some Linux utilities such as — ping and sh :
>> docker exec -it nginx-container bash
# ping
root@b3d4f894386b:~ apt update && apt upgrade
root@b3d4f894386b:~ apt-get install iputils-ping
root@b3d4f894386b:~ ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=110 time=0.537 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=110 time=0.427 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=110 time=0.531 ms
#-------------------------------------------------------
# sh
root@b3d4f894386b:~ sh
/ echo "test"
test
#-------------------------------------------------------
# add a new user and change the owner of the etc directory
root@b3d4f894386b:~ useradd test
root@b3d4f894386b:~ chown test:test etc
root@b3d4f894386b:~ ls -la | grep etc
drwxr-xr-x 1 test test 4096 Nov 16 02:23 etc # etc directory's owner changed to "test" user
With a Custom AppArmor Profile
We can use the following custom AppArmor Profile for the nginx-container , it will help us to block ping and sh and many other capabilities:
#include <tunables/global>
profile docker-nginx flags=(attach_disconnected,mediate_deleted) {
#include <abstractions/base>network inet tcp,
network inet udp,
network inet icmp,
deny network raw,
deny network packet,
file,
umount,
deny /bin/** wl,
deny /boot/** wl,
deny /dev/** wl,
deny /etc/** wl,
deny /home/** wl,
deny /lib/** wl,
deny /lib64/** wl,
deny /media/** wl,
deny /mnt/** wl,
deny /opt/** wl,
deny /proc/** wl,
deny /root/** wl,
deny /sbin/** wl,
deny /srv/** wl,
deny /tmp/** wl,
deny /sys/** wl,
deny /usr/** wl,
audit /** w,
/var/run/nginx.pid w,
/usr/sbin/nginx ix,
deny /bin/dash mrwklx,
deny /bin/sh mrwklx,
deny /usr/bin/top mrwklx,
capability chown,
capability dac_override,
capability setuid,
capability setgid,
capability net_bind_service,
deny @{PROC}/* w, # deny write for all files directly in /proc (not in a subdir)
# deny write to files not in /proc/<number>/** or /proc/sys/**
deny @{PROC}/{[^1-9],[^1-9][^0-9],[^1-9s][^0-9y][^0-9s],[^1-9][^0-9][^0-9][^0-9]*}/** w,
deny @{PROC}/sys/[^k]** w, # deny /proc/sys except /proc/sys/k* (effectively /proc/sys/kernel)
deny @{PROC}/sys/kernel/{?,??,[^s][^h][^m]**} w, # deny everything except shm* in /proc/sys/kernel/
deny @{PROC}/sysrq-trigger rwklx,
deny @{PROC}/mem rwklx,
deny @{PROC}/kmem rwklx,
deny @{PROC}/kcore rwklx,deny mount,
deny /sys/[^f]*/** wklx,
deny /sys/f[^s]*/** wklx,
deny /sys/fs/[^c]*/** wklx,
deny /sys/fs/c[^g]*/** wklx,
deny /sys/fs/cg[^r]*/** wklx,
deny /sys/firmware/** rwklx,
deny /sys/kernel/security/** rwklx,
}
Save the above-defined AppArmor profile to the host system. We can save the profile in the/etc/apparmor.d/docker-nginx file. And then load the AppArmor profile using apparmor_parser command to make it available for the container.
# loading AppArmor Profile
>> apparmor_parser /etc/apparmor.d/docker-nginx
# verify
>> aa-status
----------------------------------------------------------------------------
apparmor module is loaded.
30 profiles are loaded.
30 profiles are in enforce mode.
....
docker-default
docker-nginx # <--***
....
Unload: To unload an AppArmor profile use the following command:
# [PATH TO THE PROFILE]
>> apparmor_parser -R /etc/apparmor.d/docker-nginx
Create the nginx-containeralong with the custom AppArmor profile in the detached mode, so that we can “exec” into the running container.
>> docker run --security-opt "apparmor=docker-nginx" \
-p 80:80 -d --name apparmor-nginx nginx
Now let’s try some operations to test the AppArmor Profile:
>> docker container exec -it apparmor-nginx bash
root@1bafbecdf454:/~ sh
bash: /bin/sh: Permission denied #<----
root@1bafbecdf454:/~ useradd test
Cannot open audit interface - aborting. #<----
root@1bafbecdf454:/~ apt update && apt upgrade
...
W: chmod 0700 of directory /var/lib/apt/lists/partial failed -
SetupAPTPartialDirectory (1: Operation not permitted) #<----
...
In the above illustration, we can see the AppArmor profile making differences and blocking several actions.
AppArmor For Kubernetes
Requirements
● Container Runtime (such as — docker, containerd, cri-o, etc) must support AppArmor.
● AppArmor must be installed on every node where the pod will be scheduled.
● AppArmor profiles must be available on every node.
● AppArmor profiles are specified per container.
To specify the AppArmor profile for a container with a pod we have to add annotations to the Pod’s metadata in the following way :
container.apparmor.security.beta.kubernetes.io/<container_name>: <profile_ref>
Where <container_name> is the name of the container to apply the profile to, and <profile_ref> specifies the profile to apply. The profile_ref can be one of the:
runtime/default to apply the runtime's default profile ( such as docker-default profile)
localhost/<profile_name> to apply the profile loaded on the host with the name <profile_name>
unconfined to indicate that no profiles will be loaded
Following is the illustration of how we can specify the AppArmor profile inside a pod manifest for a particular container.
apiVersion: v1
kind: Pod
metadata:
name: pod-1
annotations:
container.apparmor.security.beta.kubernetes.io/<container-name>: localhost/<profile-name>
spec:
containers:
- name: <container-name> #<----
image: nginx
Walkthrough
Suppose we have the following AppArmor profile, which will block the ‘top’ command from inside a pod.
#include <tunables/global>
profile k8s-apparmor-example-deny-write flags=(attach_disconnected) {#include <abstractions/base>
file,
deny /bin/top mrwklx,
}
To make the AppArmor profile available on all worker nodes we can manually ssh into each node and then by using theapparmor_parser we can load AppArmor Profile.
But in a production environment, there might be a bunch of worker nodes. In that case, we can use the following script to load the AppArmor profile to the nodes.
NODES=(
# The SSH-accessible domain names of your nodes
node01)
for NODE in ${NODES[*]}; do ssh $NODE 'sudo apparmor_parser -q <<EOF
#include <tunables/global>profile k8s-apparmor-example-deny-write flags=(attach_disconnected) {#include <abstractions/base>
file,
deny /bin/top mrwklx,
}
EOF'
done
Now let’s attach the AppArmor profile to a pod manifest :
apiVersion: v1
kind: Pod
metadata:
name: hello-apparmor
annotations:
container.apparmor.security.beta.kubernetes.io/hello: localhost/k8s-apparmor-example-deny-write
spec:
containers:
- name: hello
image: busybox:1.28
command: [ "sh", "-c", "echo 'Hello AppArmor!' && sleep 1h" ]
Then finally create the pod and ‘exec’ into the pod and try to execute the ‘top’ command :
>> kubectl create -f pod.yaml
>> k exec -it hello-apparmor -- sh
----------------------------------------------------------------------
/~ top
sh: top: Permission denied # It indicates that AppArmor profile operating perfectly
Next
Seccomp — Secure Computing Mode | Kubernetes | Docker
If you found this article helpful, please don’t forget to hit the Follow 👉 and Clap 👏 buttons to help me write more articles like this.
Thank You 🖤
Level Up Coding
Thanks for being a part of our community! Before you go:
- 👏 Clap for the story and follow the author 👉
- 📰 View more content in the Level Up Coding publication
- 🔔 Follow us: Twitter | LinkedIn | Newsletter
🚀👉 Join the Level Up talent collective and find an amazing job
AppArmor For Docker | Kubernetes was originally published in Level Up Coding on Medium, where people are continuing the conversation by highlighting and responding to this story.
This content originally appeared on Level Up Coding - Medium and was authored by Mohammed Shamim
Mohammed Shamim | Sciencx (2022-11-22T02:16:54+00:00) AppArmor For Docker | Kubernetes. Retrieved from https://www.scien.cx/2022/11/22/apparmor-for-docker-kubernetes/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.