The cloud native year-in-review

Every Wednesday, Nick Chase and Eric Gregory from Mirantis go over the week’s cloud native and industry news on the Radio Cloud Native podcast.

This week, Eric and regular guest-host John Jainschigg discussed:

• New numbers on Kubernetes management challenges

• Google releases open source OSV-Scanner

• Major vulnerability in AWS Elastic Container Registry discovered and fixed

• And more on the podcast, including a wide-ranging conversation about cloud native trends in 2022 and the year ahead

You can watch the entire episode below or download the podcast from Apple Podcasts, Spotify, or wherever you get your podcasts. If you'd like to tune into the next show live, follow Mirantis on LinkedIn to receive our announcement of the next broadcast.

John: Mike Vizard, writing in Container Journal a few weeks back, did some analysis on the November update to DataDog’s several-years-running Container Report, pointing up some of the management challenges Kubernetes users are confronting. The report itself, though, is pretty upbeat: Kubernetes use among container-oriented organizations grew by 5%-ish between January and June of this year. Serverless is now used by 35% of these same orgs. Multi-cloud is also growing, and scaling – as you’d expect – with organization size: smaller orgs tend to have single clouds, but about 35% of orgs with 1000+ employees have more than one.

Ingress is now used by about 35% of orgs—DataDog itself points to Kubernetes Gateway API as the new frontier here: they’re waiting to see if Gateway displaces ingress or if the two functions will be used together. Service mesh is still way early—only 10% of orgs are using Istio and another roughly 2% are using LinkerD, with others seen more rarely.

The management challenges Vizard talks about in his piece are daunting, though. Most hosts are still running Kubernetes 1.21, which reached end of life earlier this year. A lot of hosts, moreover, are using an unsupported version of containerd to run containers, and about 30% of hosts are running version 1.4, which is also past EOL. The problem is that—for some orgs—staying with a Kubernetes version has become a necessity, at least in the short term, since existing apps will break because of changes in newer releases of the platform.

Organizations are doing a little better with access management this year—only 40%, down from 50% in 2019, are configuring Kubernetes with “overly permissive RBAC privileges.” But now the improvement seems to have flattened. We saw a few new projects at Kubecon, in October, that want to improve matters by making RBAC easier to use, and finer-grained. Hopefully they’ll have some success. We’d hope that widespread use of Open Policy Agent in DevSecOps frameworks will also start making this stat fall again.

Overall, the DataDog report is pretty illuminating. Vizard spoke with John Kendall, Senior Product Manager of Containers at Datadog, who said that the way forward was with managed Kubernetes services that provide organizations access to curated instances of the platform. Working with such services, Vizard pointed out, can both help solve the update challenge – for those who decide to update K8S – and make staying with a legacy version much less risky: the managed service provider taking responsibility for securing and continually optimizing. Managed services providers may also be able to help by updating existing cloud native applications to thrive on newer versions of Kubernetes, and by helping to build software supply chains that make it easier to create apps without Kubernetes version dependencies.

Eric: Google announced a new Go-based vulnerability scanner that utilizes the OSV.dev vulnerability database, fittingly titled OSV-Scanner.

Introduced in 2021, the OSV (or Open Source Vulnerabilities) project consists of…

1. a standardized format for vulnerability data that is mapped to open source versioning schemes (and is basically just a standardized organizational pattern in JSON, so it’s human and machine readable)

2. a site and API that aggregates OSV-formatted data from distributed databases  

The new OSV-Scanner utility evaluates the whole open source dependency tree against its mega-database. The major advantage here is the breadth of package types covered, including not just Go but (deep breath) Linux kernel, Debian, Alpine, Android, npm, PyPI, crates.io, Maven, RubyGems, NuGet, Packagist, and OSS-Fuzz. Some estimates reckon OSV.dev as the largest existing vulnerability database, so it’s nice to have an automatable tool to leverage that. 

According to Google’s announcement post, the next steps for the project should bring standalone CI actions and support for C/C++ vulnerabilities.

John: Container registries are part of any container development setup, and are now critical to the broader ecosystem allowing for centralized storage and distribution of both private and public/commercial container images. And they’re not super-super complicated – there are lots of ways to improve container registry functionality, like encryption and image scanning and image signing and the ability to store artifact formats beyond container images. And there’s a vivid competitive market around such improvements. But basic registry stuff is typically imagined to be pretty solid.

Nah. AWS just fixed a severe vulnerability in their Elastic Container Registry that could have been bad news. Discovered by Gafnit Amiga, director of security research at Lightspin, the vuln would let bad people (and security researchers, okay) delete, update, and modify ECR public images, as well as fool with images stored in ECR belonging to private AWS accounts. ECR hosts many super-popular public images, including a “top six” that alone account for something like 13 billion cumulative pulls.

Amiga said that the vulnerability might allow an attacker to poison public images and then see these used widely, based on the trust created around public ECR’s commitment to verification. With essentially every detail of containers open to modification, potential attacks that could be launched via this supply chain vulnerability would be, she added: “limited only by the craftiness and goals of the adversary.”

Well, adversaries can sit back down, because the vulnerability was reported to AWS on November 12, and fixed by 24 hours later. AWS did a serious audit thereafter and are confident that the only activity associated with the issue was between accounts owned by the researcher. No other customer accounts were touched. Brava to Lightspin and this researcher. Side-note: women in cybersecurity are saving the world’s butt every day.

Check out the podcast for more of this week's stories.