This series of articles has been formerly published on my blog site and can be found here: http://luka.sh/node/124.
My first encounter with containers in the Linux edition was significantly “burdened” with the previous experiences with virtualization. The containers were another, after paravirtualization, step towards making the visualization even “lighter” and less resource consuming! I thought they were. When my colleagues were experimenting with the duet Vagrant + VirtualBox I tried to experiment with Lxc. Still, I was not especially interested in how exactly the Lxc works — it was just “a virtual environment”. A moment of “enlightenment” (some kind of) came during the “Linux Autumn 2016”. I participated there in a workshop concerning Linux Namespaces — it was fascinating! However, after the conference, I put this subject aside. Sometime in 2017 I looked through the recordings from DockerConf 2015 where I found a record called: “Cgroups, namespaces, and beyond: what are containers made from?” — it reminded me of the “Linux Autumn” conference and one of my “post-autumnal” resolutions: to look at Namespaces more closely!
Linux Namespaces (NS) is a product of the 21st century. The first namespace appeared around 2001 or 2003 (probably Wikipedia knows exactly when :)) all the currently defined NS are in Linux since the version 3.8 (so for a while now). I will not copy the information from Wikipedia here (link above — I recommend that page), for me Namespaces are mostly the mechanism thanks to which the idea of containers in the GNU/Linux systems is implemented.
How do tools such as Lxc or Docker look like in this context? If we look more closely at their operation, it will turn out that we can, to some extent, regard them as the high-level tools simplifying managing NS by defining structures called containers. Of course, it is quite a simplified image, but it is true :) After discussing NS I will show you how to “connect” with the Lxc or Docker container without using client’s Lxc/Docker but using only tools from userspace used for managing NS. At the end of this serie I will go back to this (simplified) outlook on Lxc and Docker (as the high-level tools for managing NS), however here I am going to mention a few things:
I am a Docker enthusiast! The presented outlook on containers in the context of NS is not based on the willingness to show that it can be done differently — absolutely not! This outlook is an opportunity to talk about the low-level mechanisms; no one (especially me) wants to resign from the high-level tools! (the low-level mechanisms mean a lot of problems!)
Namespaces are the mechanisms implemented in Linux, and the “containers” are an idea realized (in Linux) while using NS — which makes the Docker version known from the GNU/Linux systems in the Windows or macOS system works in the virtual machine (Virtualbox, HyperV, or HyperKit) — they have to do so this because Docker and Lxc need Linux to function ( “compiling” Docker for other system does not work by itself!)
Containers are not (theoretically) dependent on a particular supplier of the high-level solutions (Docker, Lxc) — with time there will be (probably) more implementations and projects connected to managing containers understood as the groups of namespace — see, e.g. rkt — that is why this technology seems to have a good future :) (I will write about some interesting, in my opinion, futuristic scenario in the “summary” article)
The current NS list contains seven positions (recently I heard about the 8th NS — the Time Namespace — so the situation is dynamic). Later I ‘m going to discuss 4 of them in details and I mention three others (cgroups, userId, ipc). A page in the system manual (man) mentions a configuration setting for each of the NS responsible for turning on a given NS in the Linux binary code. It means that NS (all or part of them) will not be available in Linux (> 3.8) if they have not been added on purpose in the compilation process, even though in the default, distributional compilations they are mostly added (at least in such distributions as Debian or CentOS, however it is worth mentioning that in case of CentOS we have six, not seven, NS available in the default, distributional kernel). It is easy to check which NS have been added (during compilation process) — I will show how, now I want just to add that after compiling a given NS we don’t have to “activate”, “turn on” or “start” it in any way — each compiled namespace is active from the start of the Linux operation.
What does it specifically mean? NS are connected to processes. There is no NS where no process “works” (or to which it is not attributed to). Imagine a tree of processes (similar to that which you can see after the command ps axf), process with ID = 1 (init/systemd) is started as the first one and – what is the most important – it is started with attributed id for each compiled NS. Those identifiers can be checked in few ways, let’s begin with the most "basic" one – from checking the information in the /proc directory. Without going into details on /proc (the information can be found in the Internet), in this situation it is important that every process in the system has a subdirectory in the proc folder and among different information (files and folders) in (this) process subdirectory, we can also find the folder named ns which (obviously) contains information on NS ids attributed to a given process. So the directory /proc/1/ns contains the information on NS ids with which the process with PID = 1 has been started (and because the process with PID = 1 is the first process started in the system, NS attributed can be treated as “default”):
$ sudo ls -l /proc/1/ns
total 0
lrwxrwxrwx 1 root root 0 Aug 4 18:18 cgroup -> cgroup:4026531835
lrwxrwxrwx 1 root root 0 Aug 4 18:18 ipc -> ipc:4026531839
lrwxrwxrwx 1 root root 0 Aug 4 18:18 mnt -> mnt:4026531840
lrwxrwxrwx 1 root root 0 Aug 4 18:18 net -> net:4026531957
lrwxrwxrwx 1 root root 0 Aug 4 18:18 pid -> pid:4026531836
lrwxrwxrwx 1 root root 0 Aug 4 18:18 user -> user:4026531837
lrwxrwxrwx 1 root root 0 Aug 4 18:18 uts -> uts:4026531838Each process is connected with exactly one NS of a given type (above you can see that in this system all 7 NS have been “compiled”). There is no possibility to have a given process connected to two NS of the same type. NS (their identifiers) are inherited after the parent process unless while starting a process, we explicitly indicate that we want to create/use new NS for the sake of this particular process.
This will surely, based on the examples (next articles), be presented much simpler.
One more notice: the name of the Namespace refers both to the particular NS (particular identifiers, which there can be plenty of because there can be many defined NS — many identifiers — as part of each of 7 defined types) or to the “type” itself (there are currently 7 of them) — it does not make it simple, and we need to look at the context.
How to check which NS we have at our disposal? Of course, checking the ns directory for any process is a solution, but we have also quite a helpful command:
$ lsnslsns can display a list of namespaces assigned to a given process. Because, as I wrote, each process has an attributed identifier for each of the available NS, the list will show exactly which NS we have at disposal (so maximally 7 of different NS when all of them have been added during compilation):
$ lsns
NS TYPE NPROCS PID USER COMMAND
4026531835 cgroup 4 421 lukasz /lib/systemd/systemd --user
4026531836 pid 4 421 lukasz /lib/systemd/systemd --user
4026531837 user 4 421 lukasz /lib/systemd/systemd --user
4026531838 uts 4 421 lukasz /lib/systemd/systemd --user
4026531839 ipc 4 421 lukasz /lib/systemd/systemd --user
4026531840 mnt 4 421 lukasz /lib/systemd/systemd --user
4026531957 net 4 421 lukasz /lib/systemd/systemd --userIn the column “COMMAND” we can see a command for which a given NS has been created (so after which we inherit it, not necessarily indirectly). I need to make it clear that our current permissions influence the result of the command lsns, e.g. in the example above the process with pid 4 inherits all NS after the process from pid 1 – however, our user does not have the same information. Nevertheless, if we have a possibility to execute the command with the administrator’s privileges, we can quickly check it! Let’s begin with entering the PID identifier of the shell process in which we are currently working:
$ echo $$
1234In the shells bash/zsh the command echo $$ will return to us pid of current shell process. Now we can execute the command:
$ sudo lsns –p 1234
NS TYPE NPROCS PID USER COMMAND
4026531835 cgroup 233 1 root /sbin/init
4026531836 pid 230 1 root /sbin/init
4026531837 user 233 1 root /sbin/init
4026531838 uts 230 1 root /sbin/init
4026531839 ipc 230 1 root /sbin/init
4026531840 mnt 222 1 root /sbin/init
4026531957 net 229 1 root /sbin/initWe used the administrator’s privileges, and we received more detailed information on a process after which we inherit namespaces.
sudo lsns itself can help us display all the currently active NS, the difference between it and the previous command (without the administrator’s permissions) is that the list will include also NS defined in the context of other processes (if there are any), e.g.:
$ sudo lsns
NS TYPE NPROCS PID USER COMMAND
4026531835 cgroup 72 1 root /sbin/init
4026531836 pid 72 1 root /sbin/init
4026531837 user 72 1 root /sbin/init
4026531838 uts 72 1 root /sbin/init
4026531839 ipc 72 1 root /sbin/init
4026531840 mnt 69 1 root /sbin/init
4026531857 mnt 1 13 root kdevtmpfs
4026531957 net 72 1 root /sbin/init
4026532108 mnt 1 203 root /lib/systemd/systemd-udevd
4026532156 mnt 1 317 systemd-timesync /lib/systemd/systemd-timesyncdAbove we can see that in the Debian GNU/Linux system the process kdevtmpfs (pid 13), and the processes started with pid 203 and 317 – define their "own" NS "mnt". By checking e.g. process 317, we see that this process, outside NS "mnt" (defined specially for it) uses "default" NS (defined for the process with pid 1):
$ sudo lsns -p 317
NS TYPE NPROCS PID USER COMMAND
4026531835 cgroup 72 1 root /sbin/init
4026531836 pid 72 1 root /sbin/init
4026531837 user 72 1 root /sbin/init
4026531838 uts 72 1 root /sbin/init
4026531839 ipc 72 1 root /sbin/init
4026531957 net 72 1 root /sbin/init
4026532156 mnt 1 317 systemd-timesync /lib/systemd/systemd-timesyncdLet’s look at each NS a bit closer! Next week — mnt namespace!
