System Containers: Summit 2016
Colin Walters, Platform Engineering, Red Hat, Inc.
keybase.io/walters | walters@{redhat.com,verbum.org}
FOSS and the community
Don't take it for granted
First, there was the atom
Project Atomic: Encompasses many container efforts, including Atomic Host, Docker, Kubernetes, Atomic App, a Vagrant development box "ADB", etc.
OpenShift v3: Next-gen PaaS based on Docker and Kubernetes
Both of these drive technology into Fedora and CentOS
Project Atomic Host
- Atomic Host - rpm-ostree + docker + Docker partitioning
- yum install docker
Docker containers
Extending the model with Docker formatted containers
atomic run centos/tools
Tracing/debug tools live in container, but operate on host
All traces can be easily removed
Downsides: gdb
atomic install
Setup code can mostly live in container (see also Ansible)
Powerful: Can `chroot host` (but also very privileged, RPM equiv)
No mechanism for dependencies on host
Container runs via docker
In the ecosystem
Many management layers
Kubernetes DaemonSet
Iterating towards 4-5 layers
Atomic Host
System Extensions: Layered RPMs
System Containers
Cluster Service Containers
Cluster App Containers (or just "apps")
A brief aside on development
System Containers
# atomic pull gscrivano/spc-etcd
# atomic images
# atomic install --system --name=etcd gscrivano/spc-etcd
# runc exec etcd etcdctl set /atomic.io/network/config \
'{"Network":"172.17.0.0/16"}'
System Containers Details
Uses runc, not docker daemon to run
...but still Docker/OCI format on the wire
No arbitrary install script, controlled interaction with host (no `/host`)
Not privileged, but also currently no SELinux (runs as `unconfined_service_t`)
Read-only rootfs, writable state in `/var/lib`
System Containers Storage: OSTree
Docker (layers of tarballs) on the wire, but imported into OSTree
# ostree refs ociimage/236608c7b546e2f4e7223526c74fc71470ba06d46ec82aeb402e704bfdee02a2 ociimage/8f13ddfd66c2f0104d795acdf74a0b8044d464f77c4e2ed931abe929eb1a140e ociimage/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 ociimage/gscrivano/spc-etcd-latest ... centos-atomic-continuous:centos-atomic-host/7/x86_64/devel/continuous
Look at /var/lib/containers
rpm-ostree package layering
Now finally in git master (and hence CAHC)
# rpm-ostree pkg-add NetworkManager-team ... Added: NetworkManager-team-1:1.0.6-30.el7_2.x86_64 jansson-2.4-6.el7.x86_64 libnl3-cli-3.2.21-10.el7.x86_64 libteam-1.17-6.el7_2.x86_64 teamd-1.17-6.el7_2.x86_64 Run "systemctl reboot" to start a reboot
"System Extensions"
Code that runs in host context, particularly with CAP_SYS_ADMIN
- NetworkManager-{team,wifi}
- pam-yubikey
- kernel modules
pkg-add compatibility status
Doesn't yet support *replacing* base layer (incl. kernel)
Not all %post scripts are going to be compatible
e.g. nothing with custom SELinux policy
Still some work to do for daemons that adduser
pkg-add and %pre
# rpm-ostree pkg-add origin
error: Package 'origin' has (currently) unsupported script of type '%prein'
pkg-add wants live updates
Back to those 4-5 layers
Atomic Host
System Extensions: Layered RPMs
System Containers
Cluster Service Containers
Cluster App Containers (or just "apps")
Cluster service containers - Kube DaemonSets
Rely on Kube but provide service to kube/other containers
Profiling/fault injection
Maybe logging
Apps
All that other stuff is just to make this layer work
Target: OpenShift MustRunAsRange SCC