class: title, self-paced i share: Introduction Container Security and Hardening<br/> <!-- .nav[*Self-paced version*] --> .debug[ ``` ``` These slides have been built from commit: 00d7d7f [shared/title.md](https://git.verleun.org/training/containers.git/tree/main/slides/shared/title.md)] --- ## Introductions <!-- ⚠️ This slide should be customized by the tutorial instructor(s). --> - Hello! I am: - 👴 Marco Verleun (marco.verleun@i-share.nl) - 🏢 Employed by i-share (www.i-share.nl) - 👷🏼‍♂️ Devops/Gitops/Cloud/Container/Cluster/Linux engineer (Pick one) - 🎯 Secure clusters (air-gapped) running secure containers -- - This presentation is partially based on the excellent work of Jérôme Petazzo. - Feel free to interrupt for questions at any time .debug[[marco/intro.md](https://git.verleun.org/training/containers.git/tree/main/slides/marco/intro.md)] --- ## Exercises - Due to time constraints there are no exercises.... .debug[[marco/intro.md](https://git.verleun.org/training/containers.git/tree/main/slides/marco/intro.md)] --- name: toc-part-1 ## Part 1 - [Introduction](#toc-introduction) .debug[(auto-generated TOC)] --- name: toc-part-2 ## Part 2 - [Which application needs to be secured](#toc-which-application-needs-to-be-secured) .debug[(auto-generated TOC)] --- name: toc-part-3 ## Part 3 - [Possible improvements](#toc-possible-improvements) .debug[(auto-generated TOC)] --- name: toc-part-4 ## Part 4 - [Other improvements](#toc-other--improvements) .debug[(auto-generated TOC)] .debug[[shared/toc.md](https://git.verleun.org/training/containers.git/tree/main/slides/shared/toc.md)] --- class: pic .interstitial[] --- name: toc-introduction class: title Introduction .nav[ [Previous part](#toc-) | [Back to table of contents](#toc-part-1) | [Next part](#toc-which-application-needs-to-be-secured) ] .debug[(automatically generated title slide)] --- # Introduction Why talk about container security/hardening? - Good containers improve system security - Bad containers do the opposite -- What about this one? `docker run --rm -it -v /:/host ubuntu:latest` .debug[[secure-containers/intro.md](https://git.verleun.org/training/containers.git/tree/main/slides/secure-containers/intro.md)] --- ## Hardening is a must nowadays - CIS Benchmarking is a good thing to do: - Checkout: https://www.cisecurity.org/benchmark/docker and https://github.com/docker/docker-bench-security - Running `docker` in a non-root environment as well. - Consider `podman` instead of `docker` .debug[[secure-containers/intro.md](https://git.verleun.org/training/containers.git/tree/main/slides/secure-containers/intro.md)] --- ## Build images as secure as possible - Build them according to the best practices as described here <https://docs.docker.com/develop/develop-images/dockerfile_best-practices/> - Be careful when building `base images`: - Due to inherentence labels are transfered from one image to the other - What if you are listed as maintainer? (Not only labels are inherited btw.) .debug[[secure-containers/intro.md](https://git.verleun.org/training/containers.git/tree/main/slides/secure-containers/intro.md)] --- ## What about exising container images? - Do we rebuild them? - Or can we change the behavior of the container without modifying the image? -- ## Yes we can - Let's see how in the next slides where we will secure a nice demo application. .debug[[secure-containers/intro.md](https://git.verleun.org/training/containers.git/tree/main/slides/secure-containers/intro.md)] --- class: pic .interstitial[] --- name: toc-which-application-needs-to-be-secured class: title Which application needs to be secured .nav[ [Previous part](#toc-introduction) | [Back to table of contents](#toc-part-2) | [Next part](#toc-possible-improvements) ] .debug[(automatically generated title slide)] --- # Which application needs to be secured - It is a DockerCoin miner! 💰🐳📦🚢 -- - No, you can't buy coffee with DockerCoin -- - How dockercoins works: - generate a few random bytes - hash these bytes - increment a counter (to keep track of speed) - repeat forever! -- - DockerCoin is *not* a cryptocurrency (the only common points are "randomness," "hashing," and "coins" in the name) .debug[[secure-containers/sampleapp.md](https://git.verleun.org/training/containers.git/tree/main/slides/secure-containers/sampleapp.md)] --- ## DockerCoin in the microservices era - The dockercoins app is made of 5 services: - `rng` = web service generating random bytes - `hasher` = web service computing hash of POSTed data - `worker` = background process calling `rng` and `hasher` - `webui` = web interface to watch progress - `redis` = data store (holds a counter updated by `worker`) - These 5 services are visible in the application's Compose file, [docker-compose.yml]( https://github.com/jpetazzo/container.training/blob/main/dockercoins/docker-compose.yml) .debug[[secure-containers/sampleapp.md](https://git.verleun.org/training/containers.git/tree/main/slides/secure-containers/sampleapp.md)] --- ## How dockercoins works - `worker` invokes web service `rng` to generate random bytes - `worker` invokes web service `hasher` to hash these bytes - `worker` does this in an infinite loop - every second, `worker` updates `redis` to indicate how many loops were done - `webui` queries `redis`, and computes and exposes "hashing speed" in our browser *(See diagram on next slide!)* .debug[[secure-containers/sampleapp.md](https://git.verleun.org/training/containers.git/tree/main/slides/secure-containers/sampleapp.md)] --- class: pic  .debug[[secure-containers/sampleapp.md](https://git.verleun.org/training/containers.git/tree/main/slides/secure-containers/sampleapp.md)] --- class: pic .interstitial[] --- name: toc-possible-improvements class: title Possible improvements .nav[ [Previous part](#toc-which-application-needs-to-be-secured) | [Back to table of contents](#toc-part-3) | [Next part](#toc-other--improvements) ] .debug[(automatically generated title slide)] --- # Possible improvements - Useful commands when trying to improve images: ```bash docker logs docker diff docker exec ... trivy grype ``` .debug[[secure-containers/possible-improvements.md](https://git.verleun.org/training/containers.git/tree/main/slides/secure-containers/possible-improvements.md)] --- ## What do we know about the app? - The app is running ~~fine~~ - It does what it is supposed to do at least -- - Some improvement areas: - All containers are running with root privileges - Too many privileges (`man 7 capabilities`) - Root filesystem is writeable -- - Quite a few CVE's (As a result of the build) .debug[[secure-containers/possible-improvements.md](https://git.verleun.org/training/containers.git/tree/main/slides/secure-containers/possible-improvements.md)] --- ## How to improve the `docker-compose.yml` file - Let's pretend we do not want to modify the original `docker-compose.yml` file -- - Changes here might cause issues with upstream updates. - Hard to keep track of differences with merge conflicts -- - Let's override values with `docker-compose.override.yml` ```bash ❯ docker-compose ls NAME STATUS CONFIG FILES dockercoins running(5) docker-compose.yml,docker-compose.override.yml ``` .debug[[secure-containers/possible-improvements.md](https://git.verleun.org/training/containers.git/tree/main/slides/secure-containers/possible-improvements.md)] --- ## Two files are merged together `docker-compose.yml` ```yaml version: "2" services: rng: build: rng ports: - "8001:80" ``` `docker-compose.override.yml` ```yaml version: "2" services: rng: user: "4242:4242" ``` .debug[[secure-containers/possible-improvements.md](https://git.verleun.org/training/containers.git/tree/main/slides/secure-containers/possible-improvements.md)] --- ## Combined result - As shown by `docker-compose config` ```yaml version: "2" services: rng: build: rng ports: - "8001:80" user: "4242:4242" ``` .debug[[secure-containers/possible-improvements.md](https://git.verleun.org/training/containers.git/tree/main/slides/secure-containers/possible-improvements.md)] --- ## Starting scenario - Default without `docker-compose.override.yml` - Containers run with `root` identity - `/tmp` is writeable - 'chown' is possible .debug[[secure-containers/possible-improvements.md](https://git.verleun.org/training/containers.git/tree/main/slides/secure-containers/possible-improvements.md)] --- ## Improvement 1 - Change user id to 4242. Redis can no longer write to `/data` if a volume exists from a previous run so we mount a writeable directory on top of it (or just remove the existing volume) ```yaml services: ... rng: user: "4242:4242" ... redis: user: "4242:4242" tmpfs: - /data ... ``` .debug[[secure-containers/possible-improvements.md](https://git.verleun.org/training/containers.git/tree/main/slides/secure-containers/possible-improvements.md)] --- ## Improvement 2 - Lets reduce capabilities to make sure no weird things can happen ```yaml ... services: rng: user: "4242:4242" cap_drop: - ALL hasher: user: "4242:4242" cap_drop: - ALL ... ``` - It does not show anything different... .debug[[secure-containers/possible-improvements.md](https://git.verleun.org/training/containers.git/tree/main/slides/secure-containers/possible-improvements.md)] --- ## Improvement 3 - Make the root filesystem read only ```yaml ... services: rng: user: "4242:4242" cap_drop: - ALL read_only: true ``` .debug[[secure-containers/possible-improvements.md](https://git.verleun.org/training/containers.git/tree/main/slides/secure-containers/possible-improvements.md)] --- class: pic .interstitial[] --- name: toc-other--improvements class: title Other improvements .nav[ [Previous part](#toc-possible-improvements) | [Back to table of contents](#toc-part-4) | [Next part](#toc-) ] .debug[(automatically generated title slide)] --- # Other improvements - What about security of the code / packages? - Choosing the proper build image might make a big difference: - Look at the following two `Dockerfiles`: ```bash FROM python:latest RUN pip install redis RUN pip install requests COPY worker.py / CMD ["python", "worker.py"] ``` ```bash FROM python:alpine RUN pip install redis RUN pip install requests COPY worker.py / CMD ["python", "worker.py"] ``` .debug[[secure-containers/cve.md](https://git.verleun.org/training/containers.git/tree/main/slides/secure-containers/cve.md)] --- ## The only difference is the base image. - But the end result is quite different: | Base image | Image size | CVE's | |---|---|---| | python:latest | 888 Mb | 1114 | | python:alpine | 73,5 Mb | 3 | (Scanning is done with `grype` and since the writing of this presentation the numbers have increased) .debug[[secure-containers/cve.md](https://git.verleun.org/training/containers.git/tree/main/slides/secure-containers/cve.md)] --- ## The 'full' python image has even `gcc` included.... ```bash docker run --rm --entrypoint gcc -it python-latest:latest gcc: fatal error: no input files compilation terminated. ``` ```bash docker run --rm --entrypoint gcc python-alpine:latest docker: Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "gcc": executable file not found in $PATH: unknown. ``` .debug[[secure-containers/cve.md](https://git.verleun.org/training/containers.git/tree/main/slides/secure-containers/cve.md)] --- ## Consider a `multistage build` ```bash FROM python:latest as builder RUN pip install redis RUN pip install requests # Do many other heavy things here COPY worker.py / CMD ["python", "worker.py"] FROM python:alpine as runner # Copy file from previous build container COPY --from=builder /worker.py . RUN pip install redis RUN pip install requests # Change UID here USER 1234 CMD ["python", "worker.py"] ``` - The required code could be build in image 1 and transfered to image 2 leaving the majority of CVE's behind... .debug[[secure-containers/cve.md](https://git.verleun.org/training/containers.git/tree/main/slides/secure-containers/cve.md)] --- ## Result multistage build | Base image | Image size | CVE's | |---|---|---| | python:latest | 888 Mb | 1114 | | python:alpine | 73,5 Mb | 3 | | python:multistage | 73,5 Mb | 3 | .debug[[secure-containers/cve.md](https://git.verleun.org/training/containers.git/tree/main/slides/secure-containers/cve.md)]