What's New​

Helm v3 has served the Kubernetes community well for many years. During that time we saw new ways to use Helm, new applications installed via charts, the rise of Artifact Hub, and numerous tools that build on top of Helm. We also saw where we wanted to add features but the internal architecture of Helm didn't provide a path forward without breaking public APIs in the SDK. Helm 4 makes those changes to enable new features now and into the future.

Some of the new features include:

• Redesigned plugin system that supports Web Assembly based plugins
• Post-renderers are now plugins
• Server side apply is now supported
• Improved resource watching, to support waiting, based on kstatus
• Local Content-based caching (e.g. for charts)
• Logging via slog enabling SDK logging to integrate with modern loggers
• Reproducible/Idempotent builds of chart archives
• Updated SDK API including support for multiple chart API versions (new experimental v3 chart API version coming soon)

You can learn about more of the changes in the Helm 4 Overview.

Helm v3 Support​

When a major version of software comes out, it takes awhile to make the transition. Helm v3 will continue to be supported to enable a clean transition period. The dates of continued support are:

• Bug fixes until July 8th 2026.
• Security fixes until November 11th 2026.

Helm releases updates on Wednesdays (typically the 2nd Wednesday in a month) and these dates correspond with release schedule dates. During this time there will be NO features backported other than updates to the Kubernetes client libraries that enable support of new Kubernetes versions.

Learn More​

You can learn about the Helm changes in the overview or find all the changes in the full changelog. The documentation shares many more details as you can find all the ways Helm has stayed the same and the new features you can take advantage of.

Helm Booth in Project Pavilion​

Don't miss out on meeting our project maintainers at the Helm booth - we'll be hanging out the second half of each day. Drop by to ask questions, learn about what's in Helm 4, and pick up special Hazel swag celebrating Helm's 10th birthday!

• TUES Nov 11 @ 03:30 PM - 07:45 PM ET

• WED Nov 12 @ 02:00 PM - 05:00 PM ET

• THUR Nov 13 @ 12:30 PM - 02:00 PM ET

LOCATION: 8B, back side of Flux and across from Argo

Tuesday November 11, 2025​

Simplifying Advanced AI Model Serving on Kubernetes Using Helm Charts​

TIME: 12:00 PM - 12:30 PM ET

LOCATION: Building B | Level 4 | B401-402

SPEAKERS: Ajay Vohra & Caron Zhang

The AI model serving landscape on Kubernetes presents practitioners with an overwhelming array of technology choices: From inference servers like Ray Serve and Triton Inference Server, inference engines like vLLM, and orchestration platforms like Ray Cluster and KServe. While this diversity drives innovation, it also creates complexity. Teams often prematurely standardize on limited technology stacks to manage this complexity.

This talk introduces an innovative Helm-based approach that abstracts the complexity of AI model serving while preserving the flexibility to leverage the best tools for each use case. Our solution is accelerator agnostic, and provides a consistent YAML interface for deploying and experimenting with various serving technologies.

We'll demonstrate this approach through two concrete examples of multi-node, multi-accelerator model serving with auto scaling: 1/ Ray Serve + vLLM + Ray Cluster, and 2/ LeaderWorkerSet + Triton Inference Server + vLLM + Ray Cluster + HPA.

Wednesday November 12, 2025​

Maintainer Track: Introducing Helm 4​

TIME: 11:00 AM - 11:30 AM ET

LOCATION: Building C | Level 3 | Georgia Ballroom 1

SPEAKERS: Matt Farina & Robert Sirchia (Helm Maintainers)

The wait is over! After six years with Helm v3, Helm v4 is finally here. In this session you'll learn about Helm v4, why there was 6 years between major versions (from backwards compatible feature development to maintainer ups and downs), what's new in Helm v4, how long Helm v3 will still be supported, and what comes next. Could that include a Helm v5?

Contribfest: Hands-On With Helm 4: Wasm Plugins, OCI, and Resource Sequencing. Oh My!​

TIME: 02:15pm - 03:30 PM ET

LOCATION: Building B | Level 2 | B207

SPEAKERS: Andrew Block, Scott Rigby, & George Jenkins (Helm Maintainers)

Join Helm maintainers for an interactive session contributing to core Helm and building integrations with some of Helm 4's emerging features. We'll guide contributors through creating Helm 4's newest enhancements including WebAssembly plugins, enhancements to how OCI content is manged, and implementing resource sequencing for controlled deployment order. Attendees will explore how to build Download/Postrender/CLI plugins in WebAssembly, develop capabilities related to changes to Helm's management of OCI content including repository prefixes and aliases, and use approaches for sequencing chart deployments beyond Helm's traditional mechanisms.

This session is geared toward anyone interested in Helm development including leveraging and building upon some of the latest features associated with Helm 4!

Helm 4 Release Party​

TIME: 06:00 PM - 09:00 PM EST

LOCATION: Max Lager's Wood-Fired Grill & Brewery

Replicated and the CNCF are throwing a Helm 4 Release Party to celebrate the release of Helm 4! Drop by for a low country boil and hang out with the Helm project maintainers for the night! See the invitation here, and don't forget to save your spot – RSVP here.

Thursday November 13, 2025​

Mission Abort: Intercepting Dangerous Deletes Before Helm Hits Apply​

TIME: 01:45 PM - 02:15 PM ET

LOCATION: Building B | Level 5 | Thomas Murphy Ballroom 4

SPEAKERS: Payal Godhani

What if your next Helm deployment silently deletes a LoadBalancer, a Gateway, or an entire namespace? We’ve lived that nightmare—multiple times. In this talk, we’ll share how we turned painful Sev1 outages into a resilient, guardrail-first deployment strategy. By integrating Helm Diff and Argo CD Diff, we built a system that scans every deployment for destructive changes—like the removal of LoadBalancers, KGateways, Services, PVCs, or Namespaces—and blocks them unless explicitly approved. This second-layer approval acts as a safety circuit for your release pipelines. No guesswork. No blind deploys. Just real-time visibility into what’s about to break—before it actually does. Whether you’re managing a single cluster or an entire fleet, this talk will show you how to stop fearing Helm and start trusting it again. Because resilience isn’t about avoiding failure—it’s about learning, adapting, and building guardrails that protect everyone.

commit ecad6e2ef9523a0218864ec552bbfc724f0b9d3d
Author: Matt Butcher <mbutcher@engineyard.com>
Date:   Mon Oct 19 17:43:26 2015 -0600

    initial add

The first commit can be found on the helm-classic Git repository where the codebase for Helm v1 is located. This is the original Helm, before it merged with Deployment Manager and was folded into the Kubernetes project.

This commit was just the beginning. Helm would be shown off at the first KubeCon, just a few weeks later. From there Helm development would take off and a community of developers and charts would follow.

Happy 10th Birthday, Helm!

Alpha Period​

With the start of September, there is a freeze on new major features for Helm v4. This begins the Alpha phase, where API breaking changes will still happen, but the focus turns to stability and making sure the existing changes work as expected.

If you're a Helm user, during this period you can test out the current capabilities and provide feedback where things aren't working as expected. Just remember, this is alpha quality software and changes are still occurring.

For Helm SDK users, now is a good time to look at the API to see if there are any concerns with the design changes along with any impacts to your efforts.

The Alpha period runs through the month of September.

Beta Period​

The beta period starts in October. At this point the focus is on stability in preparation for release. API breaking changes should be complete and the focus transitions to fixing any bugs to ensure there is a stable release.

Testers should file bugs as they encounter any issues.

Per release schedule policy, once the first beta version is available, the final 4.0.0 release date will be chosen and announced.

Release Candidates​

At the end of October, the first release candidate will be created. This represents what we think will be released as Helm v4. If there are any major issues, they will be fixed and a new release candidate will be made.

🎉 Release 🎉​

The release is planned for KubeCon + CloudNativeCon North America 2025. in mid November, which is 6 years after the release of Helm v3 and 10 years after the creation of Helm. More details on the release will come.

Update your APT key and references to the repository using the new installation instructions.

Contribfest: Expanding the Helm Ecosystem With Helm 4​

DATE: Wednesday April 2

TIME: 16:15 - 17:30 BST

LOCATION: Level 3 | ICC Capital Suite 17

Are you passionate about Helm and looking for ways to contribute? Join Helm maintainers at this ContribFest session to help shape the future of the project and its ecosystem! This session focuses on three exciting initiatives:

• Exploring the Helm Ecosystem
• Building a Triage Team
• Diving into Helm 4

Don't miss this session with Scott, Robert, and Ingy for your chance at contributing to Helm 4 efforts! For more details on the Contribfest session, check the session page for the latest updates.

Helm Maintainer Track Talk: Helm 4 You​

DATE: Thursday April 3

TIME: 16:45 - 17:15 BST

LOCATION: Level 3 | ICC Capital Suite 10-12

Join Helm maintainers Matt Farina and Andrew Block as they provide an update on the next major version of Helm, the timelines, and the features being evaluated. They will also share how the community has been inspirational in helping make Helm 4 a reality. Since Helm continues to be a crucial component in the workflows of users and enterprises worldwide, a new version of Helm is only possible thanks to the continued collaboration from the Cloud Native community.

Full details on the talk can be found on the session page.

Helm Booth in Project Pavilion​

WED April 2 @ 15:30 – 19:45 BST

THUR April 3 @ 14:00 – 17:00 BST

FRI April 4 @ 12:30 – 14:00 BST

LOCATION: 2B, near the sticker wall + back side of Backstage

Our maintainers and contributors will be staffing the Helm project booth during the second half of each day. Let us know how you're using Helm, what questions you have, and learn more about what's coming in Helm 4. We'll have Helm stickers and custom Helm tea towels to help us all remember our time in the UK!

Helm Sessions from the Community​

Aside from sessions with our Helm maintainers, there are several talks from the cloud native ecosystem that touch on Helm – be sure to check these out as well ~

• Project Lightning Talk: Kubeflow Helm Chart - Krzysztof Romanowski, Maintainer // Tuesday April 1, 2025 @ 12:34 - 12:39 BST // Platinum Suite | Level 3
• Project Lightning Talk: Stir to Combine: Creating Porter Mixins - Sarah Christoff, Maintainer // Tuesday April 1, 2025 @ 16:07 - 16:12 BST // Platinum Suite | Level 3
• Poster Session: Helmless (PS 06): Fast Serverless Deployments Without the Overhead of Kubernetes and Terraform - Michael Reichenbach, 1KOMMA5° // Wednesday April 2, 2025 @ 13:30 - 14:30 BST // Level 1 | Hall Entrance N8-N9 | Poster Pavilion
• Jaeger V2: OpenTelemetry at the Core of Modern Distributed Tracing - Jonah Kowall, Paessler // Wednesday April 2, 2025 @ 17:00 - 17:30 BST // Level 3 | ICC Capital Suite 10-12
• Into the Shopfloor: Moving Manufacturing Execution Systems To Kubernetes - Manuel Peuster & Andrei Traian Cucuruzac, Bosch Connected Industry // Friday April 4, 2025 @ 11:45 - 12:15 BST // Level 1 | Hall Entrance N10 | Room E

See everyone this week!

Join maintainers and members of the Helm community at KubeCon + CloudNativeCon North America at the Demo Theater during the booth crawl on Wednesday November 13 at 7pm MST as we review the process involved to perform a release of the latest version of Helm so that it can be made available to the entire cloud native community. Along the way, you will learn about the associated activities, individuals, and processes involved with taking source code and transforming it into a consumable artifact for all across a myriad of supported runtimes and platforms.

Whether you are a Helm aficionado or just have an interest in release engineering, this is a session that you certainly want to circle on your agendas!

Not able to make it to KubeCon + CloudNativeCon North America? Don’t fret! The session will be recorded and made available at a later date.

This release session is just one of several Helm related events taking place at KubeCon + CloudNativeCon North America 2024. A full overview can be found here.

Helm is going to be at KubeCon / CloudNativeCon North America in Salt Lake City. There will be something happening each day of the main conference, including:

• Wednesday:
  • At a booth in the project pavilion from 3:15pm - 8pm.
  • At 7pm MST we are planning to cut a release, live.
• Thursday:
  • At a project pavilion booth from 1:45pm - 5pm.
  • Session: Contribfest: Helm 4: The Next Generation of the Kubernetes Package Manager
  • Session: The Path to Helm 4
• Friday:
  • 12:30pm - 2:30pm at the project pavilion

If you want to talk with a maintainer to learn about Helm or just give feedback, the project pavilion is the perfect place to do that. If you want to learn about Helm v4 the session "The Path to Helm 4" is going to give you an overview. If you want to make your first contributions to Helm, the Contribfest session is a place to get started.

If you're going to be in Salt Lake City, we hope to see you there.

One year later, Helm 2 became unsupported.

Here we are, over 3 years since Helm 2 became unsupported. It would be expected that all users should be migrated to Helm 3 by this time. Following consensus among the Helm org maintainers, we are announcing today the official end of support for the helm-2to3 tool.

In practice, this means that Helm 2to3 will receive no more updates (not even security patches).

We strongly discourage the use of the helm-2to3 tool moving forward, as it will be receiving no future security updates or patches. We hope that it has been a useful tool to aid in the migration from Helm 2 to 3.

One of the primary tenets that the Helm project takes very seriously is its policy related to backwards compatibility. This strict adherence towards limiting impacts that could adversely affect end consumers is one of the primary reasons why Helm has become such a stable project that the community can count on. Unfortunately, Helm’s backwards compatibility policy does impose limitations to the types of changes or features that can be introduced. It is important to note that Helm is not just a CLI tool, but it is also an SDK. The Helm SDK supports an entire ecosystem of solutions that have been developed to manage Helm content. Any breaking change could negatively impact one of those integrations.

However, even with strong adherence to the types of changes that the project can make, Helm has been able to introduce new features over time since Helm 3 was released back in 2019. The first such feature, included in Helm 3.1.0, was post rendering functionality, which allows end users to customize rendered manifests before being installed or upgraded by Helm. Allowing a post rendering step in the Helm release lifecycle was a game changer for both end users and chart maintainers alike. Users no longer have to patch, fork, or manually render chart templates locally to make their custom adjustments. As a result, chart maintainers no longer need to make their charts overly complicated to fit every possible use case under the sun. Since then, minor (new feature) versions of Helm have been released on a quarterly schedule, and continue to bring ever more functionality to end users. Another important new feature introduced during Helm 3 was the support for OCI registries as a distribution method for charts. Functionality for this experimental feature shipped with Helm 3.0.0 and became a fully functional feature in 3.8.0. Users of the CLI as well as the SDK could now confidently store charts using the same tried and true method as they store the container images. And, because charts stored in container registries follow OCI standards, Helm users and chart maintainers can use many of the same tools made for container images – which continue to be improved every day – to accomplish those tasks with their Helm charts too. Helm helped bring greater standardization to the wider Cloud Native ecosystem as it was one of the first projects that made use of OCI as a storage mechanism, which has helped popularize the use of OCI artifacts by other projects. A complete list of new Helm features can be found in the release notes of each minor version here.

The success of Helm is partially related to the architectural changes that were introduced in Helm 3. Gone are the days of the server side component, Tiller (which limited the use of Helm within multi-tenant), and security conscious environments. Countless other enhancements were also introduced that set Helm version 3 apart from its predecessors.

Businessman Marcus Lemonis said it best: “If you don’t evolve, you will die”. This sentiment is very much a fact, especially in the technology industry where new tools, approaches, and architectures are introduced with each passing day. It has been 5 years since the release of Helm version 3 and it has become clear, thanks to input from the community, that more impactful changes need to be made to the project so that Helm can continue to serve as an efficient package manager for Kubernetes. That being said, the Helm community is excited to announce the initial kickoff to pave a path toward Helm version 4.

Helm 4 ContribFest at KubeCon EU 2024​

It’s often asked: “Is Helm Popular?”. Based on responses from attendees at events, like KubeCon, who fill sessions relating to the project to maximum capacity, the answer continues to remain a resounding “YES”. At the 2024 KubeCon EU in Paris, the first steps towards soliciting feedback from the Open Source community regarding the next version of Helm occurred during the ContribFest in a session titled, “Building the Helm 4 Highway”. Clearly, there is an interest in evolving the capabilities that are part of the Helm project. Not only was the session full, but a number of compelling ideas were shared including adding support for additional templating languages other than golang, expanding the use of plugins, and increasing the level of support surrounding the secure software supply chain, such as additional methods of signing charts. The full list of ideas and topics from the ContribFest session can be found here.

Getting Involved​

For those interested in playing a role in the next version of Helm (we’d love to have you), there are several different ways to participate!

1. Join the weekly Helm 4 Roadmap meeting Fridays at 19:00 UTC where members of the community share ideas, develop solutions, and collaborate surrounding the next version of Helm.
   1. Zoom Meeting.
2. Participate in discussions in the #helm-dev channel on the Kubernetes Slack, where members of the Helm community collaborate on Helm development efforts, including those focused on Helm version 4.
3. Submit an issue within the Helm GitHub repository.
4. Follow @HelmPack on Twitter/X for project updates.

Helm helped define how to package and manage software for Kubernetes. But let’s not stop there - as we update and develop new features and capabilities in Helm 4, Helm will continue to be a tool that the community can continue to leverage confidently to find, share, and run software on Kubernetes.

Not A Vulnerability In Helm​

The Helm project rejects this disclosure’s assertion of a vulnerability within Helm.

The advisory listing on GitHub lists the categorization as CWE-200, which is for “Exposure of Sensitive Information to an Unauthorized Actor”. The documentation for this CWE notes that its use is discouraged due to frequent misuse. The description aligns with the implementation within Helm in this situation.

The --dry-run flag, when used with helm install and helm upgrade, is designed to send all of the generated chart manifests to standard out that would normally be sent to the Kubernetes API. This is useful for multiple use cases, such as debugging the development of a chart (i.e. package). These manifests are generated from the chart logic. Kubernetes Secrets have been designed by the Kubernetes project to be manifests, like any other resource. When generating manifests, including Secrets, there are times they need to be debugged in order to ensure the generated structure is correct.

This functionality, used in many situations including local development environments, is not an issue. The output is not exposed to an unauthorized actor. In fact, the output is needed. If this functionality is used in an environment that captures the output for unauthorized actors, such as in a CI system, then the vulnerability is in the use of a tool that outputs this information, rather than in the tool itself.

Consider a simple alternative situation. There is a Kubernetes manifest file containing both secret and non-secret information. kubectl, the Kubernetes CLI, is used in an attempt to apply the manifest to a cluster and the attempt fails. So, the cat utility is used to display the contents of the file containing the Secret. Is it the vulnerability in cat for displaying the information or in the CI setup for using cat to display secret information? The vulnerability is in the use of the tool rather than the tool itself.

Helm Changes​

To provide greater clarity, the Helm maintainers have made two changes.

1. Documentation has been updated to make the possible exposure of secret information explicit.
2. A flag has been added, which will be available in the next minor release, that will hide secret information in the dry-run output of these commands.

Why A Delay In Response​

When the CVE listing was released and the Helm maintainers became aware, the team went to work evaluating the potential impacts.

The Helm project takes security seriously. Some examples of this include:

• A documented security policy that enables someone to contact the security team with encrypted communications.
• Two 3rd party security audits (on versions of Helm that contained this functionality).
• A fuzzing audit and continuous fuzzing has been set up.
• Documented security assurance case, which is an OpenSSF best practices requirement on the way to a silver level.
• Helm releases are signed by the maintainer who produced the release.

Normally, a discussion about a security issue would stay internal to the Helm project. But, this listing was public and contentious. Helm maintainers contacted security professionals from multiple organizations and sought feedback on the situation prior to publicly commenting or performing any remediation work.

Please Responsibly Contact The Helm Security Team​

The Helm security team is responsive. In this calendar year, there have been releases to address two CVEs responsibly reported. You can learn more about reporting security issues in the documented process and participate, when necessary, to keep the Helm project and community secure.

Dry-run & Template Can Connect To Servers​

The dry-run feature on install and upgrade, and Helm template has not been able to communicate with Kubernetes servers. This is for security and because Helm template was designed for template rendering alone.

With Helm 3.13, there is now an opt-in option to communicate with Kubernetes by setting --dry-run=server. This tells Helm to communicate with the server for gathering information but not to perform updates. This flag also works on helm template. If you use --dry-run without setting a value it works as it did before.

Helm SDK users will find a new property, named DryRunOption, that allows you to tell the SDK to communicate with the server.

Values Handling Improvements​

Values handling has had a number of bugs filed about it. For example, importing values was inconsistent. Some of the handling even depended on ordering. Using null to remove properties was inconsistent, too. It worked in some cases but not others. That's fixed in Helm 3.13.

The order you can expect for a value to be used is:

1. User specified values (e.g CLI)
2. Imported values from dependencies
3. Parent chart values
4. Sub-chart values

JSON Indexes​

Helm repositories have an index in YAML containing details about the charts and versions of charts it contains. When this file grows large it can be expensive (e.g., processing time and memory) to parse. This is in part because YAML has anchors and aliases which are handy but cause more work to deal with.

Those index.yaml files can now contain JSON instead of YAML. Helm will generate them when the --json flag is set. The index.yaml file is used so the same file location can continue to be used for backwards compatibility. The structure for the data is the same. Helm going all the way back to 3.0.0 can handle parsing index.yaml files with JSON instead of YAML.

Tests were run on very large indexes to look at the perform difference. The results found that:

• Helm 3 versions before 3.13:
  • Parsed JSON in ~80% the time of parsing YAML
  • Used ~93% the memory parsing JSON compared to YAML
• Helm 3.13+, with special case handling to detect and handle JSON:
  • Parsed JSON in ~13% the time of parsing YAML
  • Used ~5% the memory parsing JSON compared to YAML

Get Metadata Command​

helm get provides the ability to get information about a release in a cluster. It has been able to get values, notes, hooks, and the generated manifests. In addition to those it can now get information about the metadata of the chart the release is based on.

To illustrate this, I installed WordPress and retrieved the metadata:

$ helm get metadata wp
NAME: wp
CHART: wordpress
VERSION: 17.1.13
APP_VERSION: 6.3.1
NAMESPACE: default
REVISION: 1
STATUS: deployed
DEPLOYED_AT: 2023-09-28T16:28:30-04:00

And More...​

These are just some of the highlights. Helm 3.13 includes even more features. You can read the details in the release notes.

OCI Artifacts​

Even though the majority of content stored within OCI registries are container images, additional content types can also be stored. The Open Container Initiative (OCI) defines these other assets as “Artifacts” and the ability to make use of OCI registries to store arbitrary content types has fundamentally changed how content is distributed. Instead of requiring a separate repository management tool for each type of asset, the very same registry that is already present to house container images can be reused without requiring additional software or infrastructure resources. This not only simplifies operational concerns from a hosting perspective, but provides a more uniform method of distributing content.

Helm has taken full advantage of the benefits of storing charts as OCI artifacts as it not only eliminates the complexity of managing a traditional chart repository, but simplifies the amount of resources that need to be maintained. When using a chart repository as a distribution and hosting solution, multiple assets need to be maintained:

• The packaged chart
• An optional provenance file
• A repository index file

Producing a Helm chart that is stored as an OCI artifact results in an asset with all of the necessary content packaged as a single atomic unit. The structure of the OCI artifact consists of the following three (3) key resources:

• An OCI Image Config containing the contents of the Chart.yaml file
• The packaged chart within an Image Layer
• The provenance file (when included) as an Image Layer

The composition of an OCI Helm Chart and the relationship of each of the aforementioned resources can be seen by inspecting the associated OCI Image Manifest. An example is displayed below:

{
  "schemaVersion": 2,
  "config": {
    "mediaType": "application/vnd.cncf.helm.config.v1+json",
    "digest": "sha256:8ec7c0f2f6860037c19b54c3cfbab48d9b4b21b485a93d87b64690fdb68c2111",
    "size": 117
  },
  "layers": [
    {
      "mediaType": "application/vnd.cncf.helm.chart.content.v1.tar+gzip",
      "digest": "sha256:1b251d38cfe948dfc0a5745b7af5ca574ecb61e52aed10b19039db39af6e1617",
      "size": 2487
    },
    {
      "mediaType": "application/vnd.cncf.helm.chart.provenance.v1.prov",
      "digest": "sha256:3e207b409db364b595ba862cdc12be96dcdad8e36c59a03b7b3b61c946a5741a",
      "size": 643
    }
  ]
}

This cross-section illustrates not only the components of an OCI based Helm Chart, but OCI based content in general. Each resource, whether it be a layer or a Config Manifest contain the same sets of properties within the Image Manifest.

• Size – Content size
• Digest – Content addressable reference signifying the algorithm and hash used
• Media Type – Media Type of the referenced content

The Media Type property is arguably the most significant property given the purpose of this discussion. Now that we see where they are referenced within OCI Artifacts, let’s review Media Types in further detail.

Media Types and Their Significance​

Media Types, previously known as MIME (Multipurpose Internet Mail Extensions) Types have been around since the early days of the World Wide Web era as they identify file formats and content transmitted through the internet. Their structure consists of two concise sections separated by a slash (/): types and subtypes. Only a limited number of types are defined (10 in total) as they represent a broad use of the type itself. Subtypes are much more diverse and they are organized into a tree structure to enable a hierarchy of related types.

A common Media Type that anyone who has experience with RESTful based communication is familiar with is application/json as it is used to indicate that the content being transmitted is in JSON format. By indicating a Media Type whenever content is being transferred, the format and its representation is understood by both producers and consumers.

The OCI Image Specification contains several Media Types that represent the various structural elements of an image. These include:

• application/vnd.oci.image.index.v1+json - Index Image
• application/vnd.oci.image.manifest.v1+json - Image Manifest
• application/vnd.oci.image.layer.v1.tar+gzip - gzip compressed Layer

These provide a concrete example to aid in understanding the structural composition of a Media Type. Each of these fall within the application type as they are utilized by specific applications. The subtree here can be broken down into the following components:

• vnd - The vendor tree which are associated with specific products
• oci - The organization or company responsible
• image - A short name for the type
• index/manifest/layer - an optional subcomponent of the type
• v1 - enables the versioning of the schema
• json/tar+gzip - Optional format signifier

A more detailed description of the composition of a Media Type can be found here.

Helm Media Types​

The Helm community has worked with the IANA to register three new Media Types representing each of the components comprising an OCI based Helm Chart.

• application/vnd.cncf.helm.config.v1+json
• application/vnd.cncf.helm.chart.content.v1.tar+gzip
• application/vnd.cncf.helm.chart.provenance.v1.prov

Similar to the OCI Media Types shown in the prior section, the Helm Media Types are also located within the vendor tree and use the abbreviation for the Cloud Native Computing Foundation (cncf) as the responsible organization. The remainder of the Media Type subtree is fairly self explanatory as it provides additional distinction to the type of content itself and the format. Anyone inspecting a resource within an OCI registry representing a Helm chart would be able to easily identify it as a Helm chart and understand how to interact with the content.

For those interested in learning more about the Helm Media Types, including the low level technical details, can view the registration details within IANA which provides a wealth of information including the intended use and lower level data specification.

Helm is certainly not alone in the use of OCI artifacts as a distribution method. Other prominent technologies, including Web Assembly (WASM) and Software Bill of Materials (SBOM’s), also leverage OCI artifacts. It is safe to say that we will see the continued adoption of OCI artifacts as a method of storing content by more and more projects and technologies moving forward.

The Future for Helm’s OCI Media Types​

As anyone who has worked in software development, especially with regards to large, complex projects, can relate to the amount of time it takes for features to be implemented. Work surrounding OCI integration was originally introduced back in Helm version 3.5 and how OCI artifacts in general are stored and managed have evolved since that time. The OCI version 1.1 image specification provides not only additional structure, but guidance on how to manage OCI artifacts. As this specification is not only finalized, but adopted and implemented by registry providers, we will continue to see an increased adoption of the technology. The Helm project and community will continue to evolve with the rest of the industry so that Chart producers and consumers have not only the best experience possible, but can best leverage the available technology.

Helm is described as the Kubernetes package manager. It helps simplify finding, sharing, and using software built for Kubernetes. Helm began as what is now known as Helm Classic, a Deis project begun in 2015 and introduced at the inaugural KubeCon. In January of 2016, the project merged with a GCS tool called Kubernetes Deployment Manager, and the project was moved under Kubernetes. Helm was promoted from a Kubernetes subproject to a full-fledged CNCF project in June 2018. Helm graduated as a CNCF project in April 2020. The CNCF annual survey of 2022 found that around 90% of companies are either using or evaluating Kubernetes, and Helm’s performance and security are important, for the continued business operations of these users.

What is fuzzing?​

Fuzzing is a technique for testing software for bugs and vulnerabilities by passing it pseudo-random data. The key idea is to write a fuzzing harness similar to a unit — or integration — test that will execute the application under test with some arbitrary input. The fuzzing engine that will run the fuzzing harness uses mutational algorithms to generate new inputs - also called "testcases" - that will cause the code under test to execute uniquely, i.e., generate inputs that trigger new code execution paths. The goal is then to observe if the code under test misbehaves in the event of any of the generated inputs. Fuzzing has been effective in uncovering reliability bugs and vulnerabilities in software for more than two decades, and open source software is increasingly adopting the technique.

Helm fuzzing overview​

In this engagement, the goal was to write a set of fuzzers that would cover a lot of the Helm codebase and integrate the setup into the open source fuzzing service OSS-Fuzz. OSS-Fuzz is a free service offered by Google for critical open source projects to run their fuzzers continuously and report any crashes. Continuous analysis is important due to fuzzing relying on genetic algorithms, which effectively means the fuzzers will improve over time, and OSS-Fuzz will run the fuzzers daily indefinitely. In addition to this, continuous analysis is crucial for capturing any regressions.

Helm is written in the Go programming language, making it safe from memory-corruptions. Fuzzing Go will find panics such as slice/index out of range, nil-pointer dereferences, invalid type assertions, timeouts, and out of memory. At the end of this engagement, nine issues were found, all but one of which were fixed. Refer to the report for a detailed breakdown of the issues.

At the end of this engagement, the fuzzers provide significant coverage of the Helm project, including critical parts such as chart handling, release storage, and repositories. To write these fuzzers, Ada Logics used go-fuzz-headers to deterministically create pseudo-random structs from the data provided by libFuzzer.

Closing thoughts​

The Helm team is thankful to CNCF for providing the opportunity to work with Ada Logics to develop new fuzzers for Helm. The CNCF takes security seriously, and it previously funded two third-party security audits for the Helm project, source code for the Helm client along with the process Helm uses to handle security and source code for the Helm client along with a threat model for the use of Helm. We want to thank the following Helm maintainers who participated in this endeavour and provided fixes where required: Matt Butcher, Martin Hickey, Matt Farina and Scott Rigby. We would also like to mention and thank the Flux maintainers, especially Paulo Gomes for collaborating on an issue. The fuzzing findings and fixes are valuable add-ons to the previous conclusions of the security audits. The Helm project has efficient test suites, and code changes are backed by tests, but the newly developed fuzzers and findings have provided significant value to the project. During the fuzzing, only nine issues were found, which revalidated the high quality of the Helm code. The Helm team can now maintain the newly developed fuzzers and build on them to continue code quality and security.

yxxhero has been a long time contributor to the Helm project over the years. With a supermajority vote from the existing maintainers, yxxhero will be joining to help with Helm documentation. With their background, yxxhero will continue to help strengthen both our simplified and traditional Chinese docs.

Cheers to yxxhero!

Helm Maintainer Track Talk: Learn about Helm and Its Ecosystem​

DATE: Thursday October 27

TIME: 11:00am - 11:35am EDT

LOCATION: Room 410 A

Maintainers Matt Farina, Karena Angell, Scott Rigby, & Andrew Block will be doing a Maintainer Track talk introducing Helm itself and then doing a deep dive into the ecosystem around building packages, along with using Helm packages in clusters.

Helm Booth in Project Pavilion​

WED OCT 26: 15:30 – 20:00 ET

THUR OCT 27: 14:00 – 17:30 ET

FRI Oct 28: 13:00 – 16:00 ET

LOCATION: opposite side of the Fuzz booth, near the LitmusChaos/Flux/Cortex/Falco booths

Our maintainers will be hanging out at our Helm project booth the second half of each day. We'd love to learn from the community how Helm is being used, what pain points exists, and what users are looking for in the future. Help us help you by sharing your thoughts in our Helm User Survey, even if you're unable to drop by the booth.

Now for the fun part – the Helm booth will be giving away 3D Helm stickers, glow-in-the-dark Helm stickers, Helm bike bells, or limited edition Helm + Carhartt beanies to stay warm while in Detroit!

With that, we'll see the Helm community soon in-person!

Why Helm Doesn't Have Tools To Do This​

You might wonder, why doesn't Helm provide tools to do this out of the box?

Helm is a package manager. We often compare it to package managers for other platforms like apt, yum, zipper, homebrew, and others. All of these projects, Helm included, keep their scope within the realm of package management.

Managing how instances of packages are run in an environment is a separate concern and one people have varying ideas about. For example, some people use Ansible, others use Terraform, some use both, and some use something entirely different. Different tools can even use different methods (e.g. some are push based and others pull based). All of these are able work with the same package managers.

The Helm project strives to provide a package manager that works well with various other tools that can use a variety of different methods to manage releases.

Declarative and Imperative​

In the Kubernetes space we talk about declarative management. If you're not familiar with the concept, here is a brief explanation.

With declarative management you declare to the system what you want the end state to look like. For example, that you want X number of instances of your workload to be running. The system then works to make this a reality and usually reports status on the progress of making the declared status a reality. Over time, the way the system makes the declared state a reality can change without the need for what you declare or the status of the progress to change.

Imperative management has to do with telling the system what to do step by step. Instead of declaring what you want you tell the system each step to take to achieve the end goal.

Kubernetes provides a means to do both declarative and imperative management of resources. As the Kubernetes community tends to prefer declarative management, when possible, the rest of this post is going to focus on declarative tools you can use with Helm.

Tools​

The Kubernetes ecosystem has produced numerous projects of various styles to help you declaratively manage your Helm releases. To illustrate the options we will look at sister projects to Helm in the Cloud Native Computing Foundation (CNCF) and some more general open source projects. You can find more options in the CNCF Landscape.

CNCF Projects​

The scope of this section is limited to graduated and incubating CNCF projects. There are over 100 CNCF projects and many of them are sandbox projects. You can learn more about the differences between types of projects in the maturity level explanation. The following projects are worth looking at:

• Flux Helm Controller - Flux is a collection of projects that enable GitOps. One of the components provides a GitOps method to manage Helm releases. Flux natively supports Helm.
• Argo CD - The Argo project defines itself as providing "Open source tools for Kubernetes to run workflows, manage clusters, and do GitOps right." Argo CD is focused on declarative continuous delivery and has the ability to work with Helm charts.

Other Projects​

There are many projects beyond the CNCF projects you can use to help you manage your Helm releases. The following set is an example and not exhaustive.

• Helmfile - A declarative spec for deploying Helm charts.
• Captain - A Helm controller.
• Terraform Helm provider - Enables you to manage Helm charts through Terraform.
• Orkestra - Built on other tools in this list, Orkestra adds a robust dependency graph for a related group of Helm releases and their subcharts, as well as a reverse DAG for specifying dependency requirements for rollbacks.
• Fleet - A GitOps tool chain that works with Kubernetes manifests, Helm charts, and Kustomize.

High-Level Tool Comparison​

There are some differences between the tools we've looked at so far. The following table provides some insight into their differences. This is not exhaustive and you should evaluate any tools you use yourself.

Note, this comparison is from when the blog post was published. Projects change over time and the feature set may change over time. You should evaluate the projects in their current state before choosing one.

Conclusion​

If you want to use a configuration manager with your Helm and Kubernetes configuration there are many choices. While the Helm project doesn't endorse one project over another, we do suggest using a configuration manager when it's appropriate.

Over the past several years container registry developers have been working on ways to store other artifacts in container registries. To facilitate this in a cross platform manner, the Open Containers Initiative (OCI) - the organization that defines the specifications for containers - released their distribution specification which allowed other "artifacts" to be stored in registries.

Common Storage​

Since OCI artifacts now makes it possible to store more than container images, you can store charts, images, and other artifacts in a single OCI registry. Sharing a common storage standard that's not specific to Helm allows greater interoperability between tools from the wider container ecosystem for security, identity and access management, and more.

Working With Charts In Registries​

The combination of OCI artifact support in a registry and new functionality within Helm provides the capability to pull and push charts to and from a registry. You can also specify charts stored in OCI as a dependency in any Chart.yaml file. The following example illustrates logging into a registry and pushing a chart:

$ helm create demo
Creating demo

$ helm package demo
Successfully packaged chart and saved it to: /tmp/demo-0.1.0.tgz

$ echo "mypass" | helm registry login r.example.com -u myuser --password-stdin
Login Succeeded

$ helm push demo-0.1.0.tgz oci://r.example.com/myuser
Pushed: r.example.com/myuser/demo:0.1.0
Digest: sha256:7ed393daf1ffc94803c08ffcbecb798fa58e786bebffbab02da5458f68d0ecb0

More detail on working with registries can be found in the Helm documentation.

The Helm SDK​

The Helm SDK, which is useful for those building tools to integrate with Helm, also includes support to work with registries programmatically. The following example illustrates pushing a chart to a registry:

package main

import (
	"fmt"
	"io/ioutil"

	"helm.sh/helm/v3/pkg/registry"
)

func check(err error) {
	if err != nil {
		panic(err)
	}
}

func main() {
	client, err := registry.NewClient()
	check(err)

	b, err := ioutil.ReadFile("demo-0.1.0.tgz")
	check(err)

	info, err := client.Push(b, "r.example.com/myuser/demo:0.1.0")
	check(err)

	fmt.Printf("Pushed: %s\n", info.Ref)
	fmt.Printf("Digest: %s\n", info.Manifest.Digest)
}

More detail can be found in the documentation for the registry package.

Limitations​

There are some limitations when using registries to store charts compared to Helm repositories or storing container images in registries.

Helm repositores can be added and searched from the local Helm client. This is similar to how repositories work with other package managers such as zypper or apt. When working with OCI registries, this is not an option. OCI based registries don't provide standard APIs to facilitate searching.

While the OCI specification provides support for artifacts, not all registries support storing Helm charts or other artifacts that are not container images. Before choosing a registry, you should confirm whether it supports storing Helm charts.

Artifact Hub Support​

Artifact Hub, another CNCF project, provides a means to search and discover cloud native assets, including charts. Helm charts stored in OCI based registries can be listed on Artifact Hub, which already knows how to work with them. More details on working with Artifact Hub and Helm charts in container registries can be found in their documentation.

ORAS​

The OCI Registry as Storage (ORAS) project, another CNCF project, is used by Helm as the underlying library for working with registries. The ORAS project bills itself as:

Registries are evolving as generic artifact stores. To enable this goal, the ORAS project provides a way to push and pull OCI Artifacts to and from OCI Registries.

If you want to work with other artifacts in registries the ORAS project may provide some tools to help you.

Thanks and Help Us Keep Improving​

Thanks to everyone who worked on adding support for Helm charts and OCI registries, from the initial experiment to bringing this to a full feature. Thanks especially for end user testing, bug reports and feature requests, and coordination between community members and maintainers across several CNCF projects.

We hope this post helps give a good intro, and some things to consider when evaluating storing your Helm charts in OCI. See if it fits your needs, take it for a spin, and let us know what you think!

Last week, the Helm project maintainers voted to elect Karen onto the Helm org maintainers board. In this role, Karen will help shape the many projects that are hosted under the Helm umbrella, bringing her insights from the wider ecosystem to her leadership of the Helm organization.

Congratulations to Karen!

This week, the Helm maintainers voted to elect Martin onto the Helm Org Maintainers board. In this new role, Martin will help shape not just Helm, but the many projects that are hosted together with Helm. As a first order of business, Martin will be migrating the Map KubeAPIs plugin to the Helm organization.

Congratulations to Martin!