We've taken contributions from dozens of people and released several revisions, one of which was adopted by Google, IBM and others with the release of Grafeas.
Below is a subsequent revision we've proposed which is being considered by the Scanning SIG and resolves some of the previous limitations with flexibility.
Kubernetes nginx ingress chart
Role Based Access Control (RBAC)
Before Kubernetes 1.6, most deployments used very permissive ABAC policies, including omnipotent service accounts. Because we were self-hosting our Kube 1.6, we had issues with some of the charts we were trying to use, including nginx-ingress, which had not had less permissive RBAC added.
We researched suggestions for RBAC permissions and implemented the consensus using patterns emerging in other charts, and took in suggestions and contributions from several people, especially Michael Goodness.
Understanding the vulnerability landscape in the container ecosystem is critical to our mission of securing the world, so we decided to put our technology to work to answer a key question: what is the current state of vulnerabilities in official Docker repositories?
The official Docker repositories consist mostly of images for open source and commercial software, are generally managed by the author or vendor, and contain the most widely used images in the Docker community. The 20 most popular images have been pulled tens of millions of times each. Increasingly, they are used as the base for distributed systems replacing some of the functionality of 3rd generation configuration management software.
Docker takes security seriously; this project focuses on the wider community and practices. While this article focuses on identifying problems, the next will propose some solutions.
On February 6th, we scanned 91 of the 133 official Docker repositories. This is every repository with a ‘latest’ tagged image consisting of a major linux distribution and a functional package manager. We used an open source scanner (vuls) that we modified, and then analyzed the data using tools we’ve built for our company, Federacy. Unfortunately, this removed alpine and static binaries because vuls currently supports neither. Scoring is CVSSv2 and priority follows the standard.
24% of Docker images tested have significant vulnerabilities (high or medium priority). 10.53% have high priority vulnerabilities.
Ubuntu images have significantly more vulnerabilities than Debian images. Debian is the least vulnerable major distribution. It has fewer moderate and high priority vulnerabilities (8%) than Ubuntu (27%). RHEL-based distribution sample size was very small (4% of all images).
Debian is the most widely used distribution — RHEL, the least, by far. Debian is the dominant base distribution among official docker repositories, and accounts for over 79% of latest-tagged images. Ubuntu accounts for ~16%, and RHEL-based distributions the remaining ~4%.
Older releases of Debian and Ubuntu have significantly more vulnerabilities. Correlating indicators are: a higher average CVSS score, more packages installed, and more updatable packages.
The number of high priority vulnerabilities in Debian 8.6 and Ubuntu 14.04 was considerably higher than in the new releases (8.7 and 16.04) and Average CVSS (v2) score shows the same.
New distribution releases had fewer packages installed and updatable. While this is not a direct indicator of vulnerabilities, it means a smaller surface area, and fewer unapplied updates.
The most common vulnerability was SSL Death Alert (CVE-2016–8610) This is a DoS-able vulnerability affecting software compiled against GnuTLS, OpenSSL, and NSS, including nginx but excluding httpd.
An attacker can flood vulnerable software with malformed SSL ALERT messages, because a new thread is not allocated to parse them. IP-based firewalls and deep packet inspection can be used to mitigate the vulnerability.
Most common vulnerability per distribution
Debian: CVE-2016–7056 (86% of images). An openssl vulnerability that could result in the compromise of a private key. However, it requires local access and the use of cache timing attacks during multiple signatures, so while a severe vulnerability, it is unlikely to impact most people. Affects most linux distributions because openssl is widely used.
RHEL: CVE-2016–1248 (50% of images). A vim vulnerability that could result in code execution and privilege escalation. Affects any system with vim, and requires opening a malicious file.
Most common high priority vulnerabilities
CVE-2016–4658 (5% of images). A libxml2 vulnerability that is misleading. Affects most linux distributions, not just Apple products as stated in the CVE, and despite being rated moderately is a significant vulnerability for people who make libxml functionality accessible remotely, which includes nokogiri and most open source software that parses XML/HTML.
CVE-2016–1252 (5% of images). A vulnerability in how the apt package manager parses signed repositories could lead to altered packages and resulting code execution. Affects Debian 8.7 and Ubuntu 16.10, 16.04, and 14.04.
As engineers we have a responsibility to our users and other stakeholders. Being compromised isn’t just embarrassing, it can cause major damage to the trust in our brands. It can also have serious financial implications, as evidenced most recently by the $350m haircut Yahoo took on their acquisition price.
Managing known vulnerabilities is the first step towards a strong security posture. If we’re not updating our systems, and keeping an eye on emerging vulnerabilities that are yet to be patched upstream, we’re basically leaving the front door wide open.
The challenge, of course, is how to best roll out updates. Many startups have invested in CI/CD systems for their code, but many have not implemented a similar process for building images or validating third-party images. Likewise, very few startups run automated vulnerability scanning in production. At Federacy, we’re trying to make this so simple and useful that it’s a no-brainer for every startup.
1. More Images. Add more sources of images, especially cloud (AWS, Digital Ocean, Google Compute, etc).
2. More Scanners. Add clair and other proprietary scanners.
3. More Time (Series). It would be interesting to see how quickly images become vulnerable after being deployed.
4. More Automation. Fully automate generation of charts and data so they are always current.
Addendum: How to Scan Docker Images and Mitigate Vulnerabilities
Scanning Docker Images became significantly easier in 2016, and we now have a number of options. Docker Hub, Quay.io, and others offer scanning for repositories they host and there are now open source scanners (clair and vuls). At Federacy, we're supporting the latter approach by making the open source scanners easier to use and connecting them to a better vulnerability database.
Mitigating vulnerabilities in Docker images is a bit more difficult, but here are a few suggestions: install package updates in your image build process. If you can, automate updating packages on your image while it is running. Add vulnerability analysis to your image build process. Use Alpine or a similar distribution. Build a static binary image.