One of the features being developed for Helm v3 is the ability to store Helm charts in OCI container image repositories (by following the distribution spec). You can think of these as repositories like Docker Hub, Quay, Azure Container Registry, and others. But, the distribution spec was designed for images rather than other artifacts which some, including more than Helm, are trying to put in there now. Work has been ongoing with the OCI to work out the best way to store these artifacts but it isn't complete and won't be complete in time to push out the 3.0.0. We don't want to delay the 3.0.0 for this feature so we are looking for feedback on some of the options we have discussed.
The options we are looking at are:
1) To document the feature as experimental in the CLI and in the documentation. Breaking changes may (likely?) happen to support the final implementation. It would not fall under Helms use of SemVer until it is no longer an experiment. We would likely not expose it via the usage information until it was ready.
2) Move the OCI integration into a plugin that would need to be installed. It would be hosted under the Helm org on GitHub. Once it is ready it would be added to Helm v3 as part of a feature release.
Note, just because Helm would be able to push and pull charts from repositories does not mean all repositories will support it. Many public repositories white list the artifacts they will store. Until Helm is added to the white list it would not work. Some repositories are not following the distribution spec and may run into issues. As use grows we hope more repositories support charts.
If you have feedback on either of the two paths we would appreciate it.
Regards,
Matt Farina
|
|
FYI... this was meant to go to the list (per Jimmy in slack) so forwarding on...
toggle quoted message
Show quoted text
----- Original message -----
From: jimmyzelinskie+cncf@...
To: Matt Farina <matt@...>
Subject: Private: Re: [cncf-helm] How to handle experiment of Helm charts in OCI repos when 3.0.0 comes out
Date: Tuesday, August 06, 2019 11:49 AM
Hey there,
I'd like to propose a third option which is to continue the existing work, but hide it from the CLI usage by default. This will allow continued development, but without user confusion. I'd argue that even if you tell people that the functionality is experimental, it'll confuse people unless they have to explicitly opt in somehow (e.g. export an environment variable).
While pulling out into a plug-in is a noble goal, App Registry was stifled a lot by the limitations of the plug-in model, which only compounded the difficulty of getting feature functionality finished. I think getting core functionality implemented first should be the priority and then a later version of Helm could focus development on pulling out as many features as possible into plug-ins.
|
|
I think what Jimmy is suggesting fits in with the first option of:
1) To document the feature as experimental in the CLI and in the
documentation. Breaking changes may (likely?) happen to support the
final implementation. It would not fall under Helms use of SemVer until it is no longer an experiment. We would likely not expose it via the usage information until it was ready.
In this case the usage would be hidden from the CLI until it's ready. I think we need to have it characterized as experimental and not following the SemVer guarantee so as not to confuse people who have used it. This way everyone is clear that could break and isn't following SemVer.
Do we hit it behind a flag of something like `--experimental`?
On Tue, Aug 6, 2019, at 11:54 AM, Matt Farina wrote:
FYI... this was meant to go to the list (per Jimmy in slack) so forwarding on...
----- Original message -----
From: jimmyzelinskie+cncf@...
To: Matt Farina <matt@...>
Subject: Private: Re: [cncf-helm] How to handle experiment of Helm charts in OCI repos when 3.0.0 comes out
Date: Tuesday, August 06, 2019 11:49 AM
Hey there,
I'd like to propose a third option which is to continue the existing work, but hide it from the CLI usage by default. This will allow continued development, but without user confusion. I'd argue that even if you tell people that the functionality is experimental, it'll confuse people unless they have to explicitly opt in somehow (e.g. export an environment variable).
While pulling out into a plug-in is a noble goal, App Registry was stifled a lot by the limitations of the plug-in model, which only compounded the difficulty of getting feature functionality finished. I think getting core functionality implemented first should be the priority and then a later version of Helm could focus development on pulling out as many features as possible into plug-ins.
|
|
Matt Fisher <matt.fisher@...>
There are a few common practices for marking features as experimental:
- Setting a field in a config file, e.g. `experimental: true` that enables all features
- A feature gates model that enables a subset of experimental or alpha features. Very similar to the first point. `feature-gates: ["oci", "fancy-feature"]`
- Appending a prefix to the command, e.g. `helm experimental-registry ...`
- The plugin model
Here's a list of projects used in production today that map to each of these features:

Matthew Fisher
Caffeinated Software Engineer
Microsoft Canada
toggle quoted message
Show quoted text
From: cncf-helm@... <cncf-helm@...> on behalf of Matt Farina via Lists.Cncf.Io <matt=mattfarina.com@...>
Sent: Tuesday, August 6, 2019 9:59 AM
To: cncf-helm@... <cncf-helm@...>
Cc: cncf-helm@... <cncf-helm@...>
Subject: Re: [cncf-helm] How to handle experiment of Helm charts in OCI repos when 3.0.0 comes out
I think what Jimmy is suggesting fits in with the first option of:
1) To document the feature as experimental in the CLI and in the documentation. Breaking changes may (likely?) happen to support the final implementation. It would not fall under Helms use of
SemVer until it is no longer an experiment. We would likely not expose it via the usage information until it was ready.
In this case the usage would be hidden from the CLI until it's ready. I think we need to have it characterized as experimental and not following the SemVer guarantee so as not to confuse people who have used it. This way everyone is clear that could break
and isn't following SemVer.
Do we hit it behind a flag of something like `--experimental`?
On Tue, Aug 6, 2019, at 11:54 AM, Matt Farina wrote:
FYI... this was meant to go to the list (per Jimmy in slack) so forwarding on...
----- Original message -----
From: jimmyzelinskie+cncf@...
To: Matt Farina <matt@...>
Subject: Private: Re: [cncf-helm] How to handle experiment of Helm charts in OCI repos when 3.0.0 comes out
Date: Tuesday, August 06, 2019 11:49 AM
Hey there,
I'd like to propose a third option which is to continue the existing work, but hide it from the CLI usage by default. This will allow continued development, but without user confusion. I'd argue that even if you tell people that the functionality is experimental,
it'll confuse people unless they have to explicitly opt in somehow (e.g. export an environment variable).
While pulling out into a plug-in is a noble goal, App Registry was stifled a lot by the limitations of the plug-in model, which only compounded the difficulty of getting feature functionality finished. I think getting core functionality implemented first
should be the priority and then a later version of Helm could focus development on pulling out as many features as possible into plug-ins.
|
|
Matt Fisher <matt.fisher@...>
The other aspect we may want to consider is the SDK. In Helm 2, we had a strict backwards compatibility contract for the Go SDK. Anything that was made public could not be changed from Helm 2.0.0 onwards. I'd like to continue that tradition when we release
Helm 3.0.0 - users have appreciated the amount of effort we put into ensuring backwards compatibility, allowing them to build other projects on top of the SDK.
In Helm 3, we use a package called `internal` which is hidden from the public SDK's view. In other words, packages made available there cannot be consumed by SDK users.
What does everyone think of moving experimental feature work into that package? That way it's hidden from SDK users while we are working out the right design. Something like `internal/experimental/<name-of-package-goes-here>`.

Matthew Fisher
Caffeinated Software Engineer
Microsoft Canada
toggle quoted message
Show quoted text
From: cncf-helm@... <cncf-helm@...> on behalf of Matt Fisher via Lists.Cncf.Io <matt.fisher=microsoft.com@...>
Sent: Tuesday, August 6, 2019 10:20 AM
To: cncf-helm@... <cncf-helm@...>; matt@... <matt@...>
Cc: cncf-helm@... <cncf-helm@...>
Subject: Re: [cncf-helm] How to handle experiment of Helm charts in OCI repos when 3.0.0 comes out
There are a few common practices for marking features as experimental:
- Setting a field in a config file, e.g. `experimental: true` that enables all features
- A feature gates model that enables a subset of experimental or alpha features. Very similar to the first point. `feature-gates: ["oci", "fancy-feature"]`
- Appending a prefix to the command, e.g. `helm experimental-registry ...`
- The plugin model
Here's a list of projects used in production today that map to each of these features:

Matthew Fisher
Caffeinated Software Engineer
Microsoft Canada
From: cncf-helm@... <cncf-helm@...> on behalf of Matt Farina via Lists.Cncf.Io <matt=mattfarina.com@...>
Sent: Tuesday, August 6, 2019 9:59 AM
To: cncf-helm@... <cncf-helm@...>
Cc: cncf-helm@... <cncf-helm@...>
Subject: Re: [cncf-helm] How to handle experiment of Helm charts in OCI repos when 3.0.0 comes out
I think what Jimmy is suggesting fits in with the first option of:
1) To document the feature as experimental in the CLI and in the documentation. Breaking changes may (likely?) happen to support the final implementation. It would not fall under Helms use of
SemVer until it is no longer an experiment. We would likely not expose it via the usage information until it was ready.
In this case the usage would be hidden from the CLI until it's ready. I think we need to have it characterized as experimental and not following the SemVer guarantee so as not to confuse people who have used it. This way everyone is clear that could break
and isn't following SemVer.
Do we hit it behind a flag of something like `--experimental`?
On Tue, Aug 6, 2019, at 11:54 AM, Matt Farina wrote:
FYI... this was meant to go to the list (per Jimmy in slack) so forwarding on...
----- Original message -----
From: jimmyzelinskie+cncf@...
To: Matt Farina <matt@...>
Subject: Private: Re: [cncf-helm] How to handle experiment of Helm charts in OCI repos when 3.0.0 comes out
Date: Tuesday, August 06, 2019 11:49 AM
Hey there,
I'd like to propose a third option which is to continue the existing work, but hide it from the CLI usage by default. This will allow continued development, but without user confusion. I'd argue that even if you tell people that the functionality is experimental,
it'll confuse people unless they have to explicitly opt in somehow (e.g. export an environment variable).
While pulling out into a plug-in is a noble goal, App Registry was stifled a lot by the limitations of the plug-in model, which only compounded the difficulty of getting feature functionality finished. I think getting core functionality implemented first
should be the priority and then a later version of Helm could focus development on pulling out as many features as possible into plug-ins.
|
|
I thought following semver on the Go API for v3 was a given. I think it's important with the way Go modules don't allow you to set a high end for the range resolution for minor versions. The tooling basically requires you to follow semver or you could break the importers experience.
If we go with feature gates I would suggest we do it as a flag or an environment variable. I can see myself setting the environment variable and using repeated commands.
toggle quoted message
Show quoted text
On Tue, Aug 6, 2019, at 1:27 PM, Matt Fisher wrote:
The other aspect we may want to consider is the SDK. In Helm 2, we had a strict backwards compatibility contract for the Go SDK. Anything that was made public could not be changed from Helm 2.0.0 onwards. I'd like to continue that tradition when we release
Helm 3.0.0 - users have appreciated the amount of effort we put into ensuring backwards compatibility, allowing them to build other projects on top of the SDK.
In Helm 3, we use a package called `internal` which is hidden from the public SDK's view. In other words, packages made available there cannot be consumed by SDK users.
What does everyone think of moving experimental feature work into that package? That way it's hidden from SDK users while we are working out the right design. Something like `internal/experimental/<name-of-package-goes-here>`.
Matthew Fisher
Caffeinated Software Engineer
Microsoft Canada
From: cncf-helm@... <cncf-helm@...> on behalf of Matt Fisher via Lists.Cncf.Io <matt.fisher=microsoft.com@...> Sent: Tuesday, August 6, 2019 10:20 AM To: cncf-helm@... <cncf-helm@...>; matt@... <matt@...> Cc: cncf-helm@... <cncf-helm@...> Subject: Re: [cncf-helm] How to handle experiment of Helm charts in OCI repos when 3.0.0 comes out
There are a few common practices for marking features as experimental:
- Setting a field in a config file, e.g. `experimental: true` that enables all features
- A feature gates model that enables a subset of experimental or alpha features. Very similar to the first point. `feature-gates: ["oci", "fancy-feature"]`
- Appending a prefix to the command, e.g. `helm experimental-registry ...`
- The plugin model
Here's a list of projects used in production today that map to each of these features:
Matthew Fisher
Caffeinated Software Engineer
Microsoft Canada
From: cncf-helm@... <cncf-helm@...> on behalf of Matt Farina via Lists.Cncf.Io <matt=mattfarina.com@...> Sent: Tuesday, August 6, 2019 9:59 AM To: cncf-helm@... <cncf-helm@...> Cc: cncf-helm@... <cncf-helm@...> Subject: Re: [cncf-helm] How to handle experiment of Helm charts in OCI repos when 3.0.0 comes out
I think what Jimmy is suggesting fits in with the first option of:
1) To document the feature as experimental in the CLI and in the documentation. Breaking changes may (likely?) happen to support the final implementation. It would not fall under Helms use of SemVer until it is no longer an experiment. We would likely not expose it via the usage information until it was ready.
In this case the usage would be hidden from the CLI until it's ready. I think we need to have it characterized as experimental and not following the SemVer guarantee so as not to confuse people who have used it. This way everyone is clear that could break
and isn't following SemVer.
Do we hit it behind a flag of something like `--experimental`?
On Tue, Aug 6, 2019, at 11:54 AM, Matt Farina wrote:
FYI... this was meant to go to the list (per Jimmy in slack) so forwarding on...
----- Original message -----
From: jimmyzelinskie+cncf@...
To: Matt Farina <matt@...>
Subject: Private: Re: [cncf-helm] How to handle experiment of Helm charts in OCI repos when 3.0.0 comes out
Date: Tuesday, August 06, 2019 11:49 AM
Hey there,
I'd like to propose a third option which is to continue the existing work, but hide it from the CLI usage by default. This will allow continued development, but without user confusion. I'd argue that even if you tell people that the functionality is experimental,
it'll confuse people unless they have to explicitly opt in somehow (e.g. export an environment variable).
While pulling out into a plug-in is a noble goal, App Registry was stifled a lot by the limitations of the plug-in model, which only compounded the difficulty of getting feature functionality finished. I think getting core functionality implemented first
should be the priority and then a later version of Helm could focus development on pulling out as many features as possible into plug-ins.
Attachments:
- Outlook-1504220459.png
- Outlook-1504220459.png
|
|
The feature gates approach sounds good to me. In general, I’m in favor of keeping the work in tree and marking experimental (vs. plugin).
@Matt when/where would the feature gates be configured? On “helm init”?
Thanks,
Josh
toggle quoted message
Show quoted text
There are a few common practices for marking features as experimental:
- Setting a field in a config file, e.g. `experimental: true` that enables all features
- A feature gates model that enables a subset of experimental or alpha features. Very similar to the first point. `feature-gates: ["oci", "fancy-feature"]`
- Appending a prefix to the command, e.g. `helm experimental-registry ...`
- The plugin model
Here's a list of projects used in production today that map to each of these features:

Matthew Fisher
Caffeinated Software Engineer
Microsoft Canada
I think what Jimmy is suggesting fits in with the first option of:
1) To document the feature as experimental in the CLI and in the documentation. Breaking changes may (likely?) happen to support the final implementation. It would not fall under Helms use of
SemVer until it is no longer an experiment. We would likely not expose it via the usage information until it was ready.
In this case the usage would be hidden from the CLI until it's ready. I think we need to have it characterized as experimental and not following the SemVer guarantee so as not to confuse people who have used it. This way everyone is clear that could break
and isn't following SemVer.
Do we hit it behind a flag of something like `--experimental`?
On Tue, Aug 6, 2019, at 11:54 AM, Matt Farina wrote:
FYI... this was meant to go to the list (per Jimmy in slack) so forwarding on...
----- Original message -----
Subject: Private: Re: [cncf-helm] How to handle experiment of Helm charts in OCI repos when 3.0.0 comes out
Date: Tuesday, August 06, 2019 11:49 AM
Hey there,
I'd like to propose a third option which is to continue the existing work, but hide it from the CLI usage by default. This will allow continued development, but without user confusion. I'd argue that even if you tell people that the functionality is experimental,
it'll confuse people unless they have to explicitly opt in somehow (e.g. export an environment variable).
While pulling out into a plug-in is a noble goal, App Registry was stifled a lot by the limitations of the plug-in model, which only compounded the difficulty of getting feature functionality finished. I think getting core functionality implemented first
should be the priority and then a later version of Helm could focus development on pulling out as many features as possible into plug-ins.
|
|
Jimmy Zelinskie <jimmyzelinskie@...>
>I think what Jimmy is suggesting fits in with the first option
Yup! I wasn't sure if the intent of the first option was to simply put the string `(experimental)` next to the usage and call it a day or not.
The Go compiler has been using environment variables for feature gating (e.g. GOMODULES111=on). Having to append `--experimental` to commands seems monotonous for those testing the functionality, although that might be desirable so that if anyone sees the bash history (or is shown the command) it's extremely clear that the work isn't stable.
toggle quoted message
Show quoted text
On Tue, Aug 6, 2019 at 12:59 PM Matt Farina < matt@...> wrote: I think what Jimmy is suggesting fits in with the first option of:
1) To document the feature as experimental in the CLI and in the
documentation. Breaking changes may (likely?) happen to support the
final implementation. It would not fall under Helms use of SemVer until it is no longer an experiment. We would likely not expose it via the usage information until it was ready.
In this case the usage would be hidden from the CLI until it's ready. I think we need to have it characterized as experimental and not following the SemVer guarantee so as not to confuse people who have used it. This way everyone is clear that could break and isn't following SemVer.
Do we hit it behind a flag of something like `--experimental`?
On Tue, Aug 6, 2019, at 11:54 AM, Matt Farina wrote:
FYI... this was meant to go to the list (per Jimmy in slack) so forwarding on...
----- Original message -----
Subject: Private: Re: [cncf-helm] How to handle experiment of Helm charts in OCI repos when 3.0.0 comes out
Date: Tuesday, August 06, 2019 11:49 AM
Hey there,
I'd like to propose a third option which is to continue the existing work, but hide it from the CLI usage by default. This will allow continued development, but without user confusion. I'd argue that even if you tell people that the functionality is experimental, it'll confuse people unless they have to explicitly opt in somehow (e.g. export an environment variable).
While pulling out into a plug-in is a noble goal, App Registry was stifled a lot by the limitations of the plug-in model, which only compounded the difficulty of getting feature functionality finished. I think getting core functionality implemented first should be the priority and then a later version of Helm could focus development on pulling out as many features as possible into plug-ins.
|
|
Matt Fisher <matt.fisher@...>
> I thought following semver on the Go API for v3 was a given.
Absolutely. I'm specifically talking about the stability of
pkg/registry, which is the package used primarily for communicating with OCI registries.
The question I'm wondering is "If we're marking certain features of the client as experimental, should the SDK receive the same treatment? If so, how?"

Matthew Fisher
Caffeinated Software Engineer
Microsoft Canada
toggle quoted message
Show quoted text
From: cncf-helm@... <cncf-helm@...> on behalf of via Lists.Cncf.Io <jimmyzelinskie=gmail.com@...>
Sent: Tuesday, August 6, 2019 10:04 AM
To: Matt Farina <matt@...>
Cc: cncf-helm@... <cncf-helm@...>
Subject: Re: [cncf-helm] How to handle experiment of Helm charts in OCI repos when 3.0.0 comes out
>I think what Jimmy is suggesting fits in with the first option
Yup! I wasn't sure if the intent of the first option was to simply put the string `(experimental)` next to the usage and call it a day or not.
The Go compiler has been using environment variables for feature gating (e.g. GOMODULES111=on). Having to append `--experimental` to commands seems monotonous for those testing the functionality, although that might be desirable so that if anyone sees
the bash history (or is shown the command) it's extremely clear that the work isn't stable.
On Tue, Aug 6, 2019 at 12:59 PM Matt Farina < matt@...> wrote:
I think what Jimmy is suggesting fits in with the first option of:
1) To document the feature as experimental in the CLI and in the documentation. Breaking changes may (likely?) happen to support the final implementation. It would not fall under Helms use of
SemVer until it is no longer an experiment. We would likely not expose it via the usage information until it was ready.
In this case the usage would be hidden from the CLI until it's ready. I think we need to have it characterized as experimental and not following the SemVer guarantee so as not to confuse people who have used it. This way everyone is clear that could break
and isn't following SemVer.
Do we hit it behind a flag of something like `--experimental`?
On Tue, Aug 6, 2019, at 11:54 AM, Matt Farina wrote:
FYI... this was meant to go to the list (per Jimmy in slack) so forwarding on...
----- Original message -----
Subject: Private: Re: [cncf-helm] How to handle experiment of Helm charts in OCI repos when 3.0.0 comes out
Date: Tuesday, August 06, 2019 11:49 AM
Hey there,
I'd like to propose a third option which is to continue the existing work, but hide it from the CLI usage by default. This will allow continued development, but without user confusion. I'd argue that even if you tell people that the functionality is experimental,
it'll confuse people unless they have to explicitly opt in somehow (e.g. export an environment variable).
While pulling out into a plug-in is a noble goal, App Registry was stifled a lot by the limitations of the plug-in model, which only compounded the difficulty of getting feature functionality finished. I think getting core functionality implemented first
should be the priority and then a later version of Helm could focus development on pulling out as many features as possible into plug-ins.
|
|
Matt Fisher <matt.fisher@...>
An environment variable seems to be the right approach. Either that or a config file similar to Docker's config.json.

Matthew Fisher
Caffeinated Software Engineer
Microsoft Canada
toggle quoted message
Show quoted text
From: cncf-helm@... <cncf-helm@...> on behalf of Matt Fisher via Lists.Cncf.Io <matt.fisher=microsoft.com@...>
Sent: Tuesday, August 6, 2019 11:53 AM
To: Matt Farina <matt@...>; jimmyzelinskie@... <jimmyzelinskie@...>
Cc: cncf-helm@... <cncf-helm@...>
Subject: Re: [cncf-helm] How to handle experiment of Helm charts in OCI repos when 3.0.0 comes out
> I thought following semver on the Go API for v3 was a given.
Absolutely. I'm specifically talking about the stability of
pkg/registry, which is the package used primarily for communicating with OCI registries.
The question I'm wondering is "If we're marking certain features of the client as experimental, should the SDK receive the same treatment? If so, how?"

Matthew Fisher
Caffeinated Software Engineer
Microsoft Canada
From: cncf-helm@... <cncf-helm@...> on behalf of via Lists.Cncf.Io <jimmyzelinskie=gmail.com@...>
Sent: Tuesday, August 6, 2019 10:04 AM
To: Matt Farina <matt@...>
Cc: cncf-helm@... <cncf-helm@...>
Subject: Re: [cncf-helm] How to handle experiment of Helm charts in OCI repos when 3.0.0 comes out
>I think what Jimmy is suggesting fits in with the first option
Yup! I wasn't sure if the intent of the first option was to simply put the string `(experimental)` next to the usage and call it a day or not.
The Go compiler has been using environment variables for feature gating (e.g. GOMODULES111=on). Having to append `--experimental` to commands seems monotonous for those testing the functionality, although that might be desirable so that if anyone sees
the bash history (or is shown the command) it's extremely clear that the work isn't stable.
On Tue, Aug 6, 2019 at 12:59 PM Matt Farina < matt@...> wrote:
I think what Jimmy is suggesting fits in with the first option of:
1) To document the feature as experimental in the CLI and in the documentation. Breaking changes may (likely?) happen to support the final implementation. It would not fall under Helms use of
SemVer until it is no longer an experiment. We would likely not expose it via the usage information until it was ready.
In this case the usage would be hidden from the CLI until it's ready. I think we need to have it characterized as experimental and not following the SemVer guarantee so as not to confuse people who have used it. This way everyone is clear that could break
and isn't following SemVer.
Do we hit it behind a flag of something like `--experimental`?
On Tue, Aug 6, 2019, at 11:54 AM, Matt Farina wrote:
FYI... this was meant to go to the list (per Jimmy in slack) so forwarding on...
----- Original message -----
Subject: Private: Re: [cncf-helm] How to handle experiment of Helm charts in OCI repos when 3.0.0 comes out
Date: Tuesday, August 06, 2019 11:49 AM
Hey there,
I'd like to propose a third option which is to continue the existing work, but hide it from the CLI usage by default. This will allow continued development, but without user confusion. I'd argue that even if you tell people that the functionality is experimental,
it'll confuse people unless they have to explicitly opt in somehow (e.g. export an environment variable).
While pulling out into a plug-in is a noble goal, App Registry was stifled a lot by the limitations of the plug-in model, which only compounded the difficulty of getting feature functionality finished. I think getting core functionality implemented first
should be the priority and then a later version of Helm could focus development on pulling out as many features as possible into plug-ins.
|
|