Understanding Linux Containers
What are Containers?
The name is derived from shipping industry.
“Developer says the code works fine on my DEV environment; Tester says no the same code doesn’t work on my System Test environment.” Sounds familiar? ♬
One of the major goal of container is to solve this problem.
Containers are collection of your application code and its dependencies packaged in a standard & lightweight unit of software. This allows you to run your application quickly and reliably on one computing environment to another. Container technology has evolved around Linux operating system features.
Containers use various Linux kernel features listed below to provided isolated environment like Virtual Machines.
- Namespaces
- CGroups
- Apparmor
- SELinux Profiles
- chroot
Linux Containers are operating system level virtualization technology which can provide multiple isolated environments on a single Linux host. Containers do not run on dedicated guest OS (Operating System) instead they share the host OS kernel. Containers are light weight & start much faster than a VM (Virtual Machine).
 |
Image credit: Docker Inc.
Note: Above VM example is Type2 (most used) hypervisor example. Type1 hypervisor (ESXi) can run directly on top of the hardware. Also, if Docker runs on WINDOWS or MAC OS host then it needs Linux VM on top of it. Docker for MAC or WINDOWS installation takes care of it.
|
Container technology is not a new phenomenon and has long been a core feature for Linux.
Google has used container technology for many years with ‘Borg’ and ‘Omega’ container cluster management platforms for running Google applications at scale. More importantly, Google has contributed to container space by implementing cgroups and participating in libcontainer projects. Google may have gained a huge gain in performance, resource utilization, and overall efficiency using containers during past years. Very recently Microsoft, who did not have an operating system level virtualization on the Windows platform took immediate action to implement native support for containers on Windows Server.
Few container standards -
1. Docker:The most popular Linux container.
2. Rocket (rkt):An emerging container standard from CoreOS (now part of RedHat).
3. Garden:The format Cloud Foundry builds using buildpacks.
4. Open Container Initiative (OCI):A Linux Foundation project developing a governed container standard.
History of Containers
Containerization technology goes all the way back to 1979. Below is short history of containers that I have collected from Wikipedia & other resources-
1979 (chroot) -It’s an UNIX operating-system system call for changing the root directory of a process and it's children to a new location in the filesystem which is only visible to a given process. The idea of this feature is to provide an isolated disk space for each process. Later in 1982 this was added to BSD.
2000 (FreeBSD Jails) - It is an operating-system system call similar to chroot, but included additional process sandboxing features for isolating the filesystem, users, networking, etc. As a result, it could provide means of assigning an IP address for each jail, custom software installations and configurations, etc.
2001 (Linux VServer) - Linux VServer is a another jail mechanism that can be used to securely partition resources on a computer system (file system, CPU time, network addresses and memory). Each partition is called a security context, and the virtualized system within it is called a virtual private server.
2004 (Solaris Containers) - Solaris Containersare combination of system resource controls and the boundary separation provided by zones. Zones act as completely isolated virtual servers within a single operating system instance.
2005 (OpenVZ) - OpenVZ is similar to Solaris Containers and makes use of a patched Linux kernel for providing virtualization, isolation, resource management, and checkpointing. Each OpenVZ container would have an isolated file system, users and user groups, a process tree, network, devices, and IPC objects.
2006 (Process Containers a.k.a. cgroups) - Process Containers was implemented at Google in year 2006 for limiting, accounting, and isolating resource usage (CPU, memory, disk I/O, network, etc.) of a collection of processes. Later on, it was renamed to Control Groups to avoid the confusion multiple meanings of the term “container” in the Linux kernel context and merged to the Linux kernel 2.6.24. This shows how early Google was involved in container technology and how they have contributed back.
2007 (Control Groups a.k.a. cgroups) - Control Groups AKA cgroups was implemented by Google and added to the Linux Kernel in 2007.
2008 (LXC) - LXC stands for LinuXContainers and it is the first, most complete implementation of Linux container manager. It was implemented using cgroups and Linux namespaces. LXC was delivered in liblxc library and provided language bindings for the API in Python3, Python2, Lua, Go, Ruby, and Haskell. Contrast to other container technologies LXC works on vanilla Linux kernel without requiring any patches. Today LXC project is sponsored by Canonical Ltd. and hosted here.
2011 (Warden) - Warden was implemented by CloudFoundry in year 2011 by using LXC at the initial stage and later on replaced with their own implementation. Unlike LXC, Warden is not tightly coupled to Linux. Rather, it can work on any operating system that can provide ways of isolating environments. It runs as a daemon and provides an API for managing the containers. Refer to Warden documentation and this blog post for more detailed information on Warden.
2013 (LMCTFY) - lmctfy stands for “Let Me Contain That For You”. It is the open source version of Google’s container stack, which provides Linux application containers. Google started this project with the intention of providing guaranteed performance, high resource utilization, shared resources, over-commitment, and near zero overhead with containers (Ref: lmctfy presentation). The cAdvisor tool used by Kubernetes today was started as a result of lmctfy project. The initial release of lmctfy was made in Oct 2013 and in year 2015 Google has decided to contribute core lmctfy concepts and abstractions to libcontainer. As a result now no active development is done in LMCTFY.
The libcontainer project was initially started by Docker and now it has been moved to Open Container Foundation.
2013 (Docker) - Docker is the most popular and widely used container management system as of May 2019. It was developed as an internal project at a platform-as-a-service company called dotCloud and later renamed to Docker. Similar to Warden, Docker also used LXC at the initial stages and later replaced LXC with it’s own library called libcontainer. Unlike any other container platform, Docker introduced an entire ecosystem for managing containers. This includes a highly efficient, layered container image model, a global and local container registry, a clean REST API, a CLI, etc. At a later stage, Docker also took an initiative to implement a container cluster management solution called Docker Swarm.
2014 (Rocket) - Rocket is a much similar initiative to Docker started by CoreOS for fixing some of the drawbacks they found in Docker. CoreOS has mentioned that their aim is to provide more rigorous security and production requirements than Docker. More importantly, it is implemented on App Container specifications to be a more open standard. In addition to Rocket, CoreOS also develops several other container related products used by Docker and Kubernetes: CoreOS Operating System, etcd, and flannel.
2016 (Windows Containers) -Microsoft also took an initiative to add container support to the Microsoft Windows Server operating system in 2015 for Windows based applications, called Windows Containers. This is to be released with Microsoft Windows Server 2016. With this implementation Docker would be able to run Docker containers on Windows natively without having to run a virtual machine to run Docker (earlier Docker ran on Windows using a Linux VM). Windows 2019 will support Kubernetes too.
As of today (May 2019) there is a significant trend in the industry to move towards containers from VMs for deploying software applications. The main reasons for this are the flexibility and low cost that containers provide compared to VMs.Container management software facilitates deployment of containers and works well with the rapid deployment strategies of the DevOps philosophy.
Docker, Rocket, and other container platforms cannot run on a single host in a production environment, the reason is that they are exposed to single point of failure. While a collection of containers run on a single host, if the host fails, all the containers that run on that host will also fail. To avoid this, a container host cluster needs to be used. One of the first most open source container cluster management platforms to solve this problem was Apache Mesos. It was initially developed at University of California, Berkeley as a research project and later moved to Apache in around year 2012. Google took a similar step to implement a cutting edge, open source container cluster management system called Kubernetes in year 2014 with the experience they got from Borg. Docker also started a solution called Docker Swarm in year 2015.
If container adoption goes the way it is right now (May 2019), it will take over virtual machines completely.Taking step back, it all started with Linux, thank to Linus Torvalds the creator of Linux OS.
The Linux Foundation (founded in 2000) is dedicated to building sustainable ecosystems around open source projects to accelerate technology development and commercial adoption. It is the home of Linux creator Linus Torvalds and lead maintainer Greg Kroah-Hartman and provides a neutral home where Linux kernel development can be protected and accelerated for years to come.
The Linux Foundation provides unparalleled support to open source communities through financial and intellectual resources, infrastructure, services, events, and training.
Today Linux is -
· Choice of OS for Top 95% of top 1 million web domain
· More than 80% of new mobile phones sold run Android OS which is based on Linux kernel.
· Top 500 super computers in the world run on Linux.
· Most global markets run on Linux e.g. New York Stock Exchange, London Exchange, NASDAQ, Tokyo Stock Exchange.
· More than 75% of cloud enabled enterprises report using Linux as primary cloud platform.
· The whole container platform ecosystem e.g. Docker, Rocket etc are all around Linux.
· Cloud Foundry
· Cloud Native Computing Foundation: Kubernetes, Rkt, gRPC, HELM, etcd and many more
· Open Container Initiative
· XGen Project
Today major cloud providers like Amazon Web Services, IBM Cloud, Microsoft Azure, Google Cloud and many more have embraced container technology to host their applications. The most popular of container software’s are – Docker, Kubernetes, Apache Mesos, rkt (a.k.a. Rocket).
Finally, Security aspects, since containers share the same host. There is concern that Containers are less secure than Virtual Machines. If there is security weakness in host kernel, it can impact all containers running on top of it. Multiple efforts have been made to tackle this problem. One of them is Docker requiring a singing infrastructure to prevent unauthorised containers from starting up. There is also container security software, such as Twistlock that profiles the behaviour of a container, and then shuts down a container that falls outside of the expected profile. There is lot more to write about container security so I will leave that for another day.
All the best in your container journey!
Comments
Post a Comment