Idea

Some of our charts create during install/upgrade secrets that are bigger than 1MB. Then Helm install/upgrades fail.

We can take some measures to remove content and cleanup charts but we can only do so much.

Our idea is to have Helm create multiple secrets to hold a Helm release.

The encoded release data would be split across multiple secrets created as needed if the encoded release data is > 1MB.

We have a working implementation of this implemented as an additional storage driver called "chunkedsecrets" which can be enabled by setting the HELM_DRIVER=chunkedsecrets environment variable.

Eventually it is better to integrate chunking into the standard secret/secrets storage driver and add automatic chunking only if the size of the encoded release data becomes too big.

Proposal

hip: 9999
title: "Support Helm release data stored in multiple K8s Secrets"
authors: [ "" ]
created: "2022-02-10"
type: "feature"
status: "draft"

Abstract

Helm release data encoded into the release field in a Kubernetes Secret can cause the Secret as a whole to exceed 1 MiB (as defined by MaxSecretSize in kubernetes/pkg/apis/core/types.go).

When this happens, the Kubernetes Secret cannot be stored in Etcd and the Helm install or upgrade fails.

Splitting the Helm release data across multiple Kubernetes Secrets resolves this problem.

Motivation

Helm release data that causes the Kubernetes Secret size to exceed MaxSecretSize will make Helm install or upgrade fail.

The existing HELM_DRIVER=secret storage driver does not handle Kubernetes Secrets become too big when storing a large encoded Helm release data.

Rationale

The logical place to support this is in the HELM_DRIVER=secret storage driver.

Specification

The feature is transparent to users.

When the HELM_DRIVER=secret storage driver notices that the resulting Kubernetes Secret will exceed MaxSecretSize then the storage driver will split the encoded release data across multiple Kubernetes Secrets.

Splitting requires additional metadata to be added to the Kubernetes Secret:

• number of chunks: integer, the number of Kubernetes Secrets that holds the release data for this Helm release version.
• chunk: integer, the chunk number. Each Secret gets a chunk number. Reconstituting the release data can be done by reading the Secret with chunk=1, then subsequently reading in numerical order any number of additional Secrets up to and including the "number of chunks".

Splitting requires the name of the Secret to include a "chunk" suffix. Example: sh.helm.release.v1.test.v1.2, 3, 4 and so on where the number after the last dot is the "number of chunk". The first Secret will not get a .1 suffix.

If no splitting is required then no additional metadata is included/set.

Backwards compatibility

If the chunking is implemented in the secret/secrets HELM_DRIVER then backward compatibility to previously installed Helm releases is guaranteed. If no chunking information is encoded in the release secret then the driver can behave as before chunking support was added. If no chunking is needed then the release secret can be created without any additional metadata and with compatible name.

Security implications

Theoretically a MTM Man in The Middle attack is possible where requests to the Kubernetes API server are intercepted. But this type of attack applies to any access to the Kubernetes API server.

How to teach this

The feature is transparent to users.

The documentation (in advanced section(s)) needs to explain that multiple Kubernetes Secrets can be created for a Helm release to counter the size limitation.

Reference implementation

We can submit a PR for this.

Rejected ideas

n/a

Open issues

https://github.com/helm/helm/issues/10986

References

n/a