Date   

Why are dependencies embedded rather than resolved?

Loritsch, Berin <bloritsch@...>
 

You'll have to forgive my ignorance on this one, but there is nothing I can see that explains this design decision--and it really makes creating multi-service deployments harder than it should be.

In almost every package management system I am used to, the dependencies are resolved.  The distinction is realized with transitive dependencies.  For example, in a C# project, I can require 2 libraries, and each of those libraries can in turn require the same logging API dependency.  The package management system figures out the version that satisfies the need for both transitive dependencies and either loads it at build time or sets up a dependency lock file for users to override if necessary.

Conversely, Helm embeds dependencies.  That means if my chart depends on another chart, that dependency is copied into my chart.  If I have two dependencies that in turn have the same dependency (like a configmap to hold a common logging configuration file) Helm will generate that transitive dependency multiple times.  Once for each dependency.  Obviously you can't have multiple configmaps with the same name.

I'm assuming the choice here was deliberate, but it is the choice that violates how I expect package management systems to behave the most.  So I find myself having to rethink everything.  Even OS level package managers like rpm and yum resolve dependencies at install time.  What is the rationale here?


--
Berin Loritsch

DOMEX Architect


7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708


Re: Pre-proposal idea: introduce a new flag for helm uninstall "--fail-on-error" and a new release status "failed-uninstall"

Joshua Packer
 
Edited

Hi Mike,

This seems like it could be very useful, especially during dev where there are more likely to be issues in the uninstall of the resources. Keeping the release would allow would give us a starting point to work from to recover, as you mention in your idea. Being an opt-in strategy, remove adverse affects for those with processes that expect the existing behavior.

Joshua Packer


Re: [EXTERNAL] Re: [cncf-helm] Is there any way to enforce constraints

Matt Fisher <matt.fisher@...>
 

So if I’m understanding correctly, you’re looking to validate that a particular resource provided by the user follows a certain schema.

 

Kubernetes resources define a schema (also known as a GroupVersionKind, or a GVK) which determines whether the resource is well-formed or not. Custom resources can be defined by the user to conform to a custom schema defined by the cluster operator.

 

In this case, I would not put the validation logic in the Helm chart, as Matt suggested. Instead, I’d recommend creating a new resource definition that conforms to the schema you desire, then when the user creates new instances of that resource, it will be validated and accepted (or rejected) by the Kubernetes API. Your Helm chart will only need to refer to the resource by its name, and the chart can safely rely on the schema’s validation logic to ensure that the correct data has been provided in the correct fields.

 

Hope that helps.

 

From: cncf-helm@... <cncf-helm@...> On Behalf Of Loritsch, Berin via lists.cncf.io
Sent: Monday, January 18, 2021 5:41 AM
To: Matt Butcher <Matt.Butcher@...>
Cc: cncf-helm@...
Subject: Re: [EXTERNAL] Re: [cncf-helm] Is there any way to enforce constraints

 

The idea is not to scan, but to validate that the specified secret is the right kind.  I.e. the user specifies `example-secret`, and the helm chart fails if the secret that exists is not the right kind... i.e. they accidentally supplied a basic-auth secret or something like that.  The ability to fail fast and provide a comprehensible error message is rather important.

 

On Fri, Jan 15, 2021 at 5:57 PM Matt Butcher <Matt.Butcher@...> wrote:

The preferred method is requiring the user to set a value that points to an existing secret. If the user points to the wrong thing, then Helm should fail to deploy that release.

 

We advise people not to write charts that try to scan for SSL certificates on clusters, as that is a security warning sign. So I would advise not using lookup​ in this (or any) case.

 

Helm is not and should not be used as a release manager tool. It is a package manager whose dependencies should be calculated up front and whose configuration should be passed by a user, not autoconfigured. In other words, think of Helm as 'apt-get', not 'chef' or 'puppet'.


From: cncf-helm@... <cncf-helm@...> on behalf of Loritsch, Berin via lists.cncf.io <bloritsch=novetta.com@...>
Sent: Friday, January 15, 2021 12:46 PM
To: cncf-helm@... <cncf-helm@...>
Subject: [EXTERNAL] Re: [cncf-helm] Is there any way to enforce constraints

 

It appears this did not garner any attention.  Is there any way to enforce constraints on a chart's dependencies?

 

For example, I would like to have my chart enforce a specific _type_ of secret the consumer provides.

 

My container can use TLS secrets and make use of them if they are provided.  However, it won't make sense if the user supplies a service account token or basic-auth token.  It would be fine if the secret is not provided at all, but if it is provided it should be a specific type.

 

I am clear on how to create the templates, and mount the secret to my container.  I'm not clear on how to enforce rather important constraints like that.

 

On Tue, Jan 12, 2021 at 10:17 AM Loritsch, Berin <bloritsch@...> wrote:

Example: I am creating a chart for a single page app server.  In it I want the ability to add a reference to the TLS secret.  The TLS secret would be provided by the person using the chart.

 

In my chart I want to enforce that the provided Secret is the type:  kubernetes.io/tls 

 

I want that constraint enforced prior to deploying the chart. 

 

--

Berin Loritsch

DOMEX Architect

7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708


 

--

Berin Loritsch

DOMEX Architect

7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708


 

--

Berin Loritsch

DOMEX Architect

7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708


Re: [EXTERNAL] Re: [cncf-helm] Is there any way to enforce constraints

Loritsch, Berin <bloritsch@...>
 

The idea is not to scan, but to validate that the specified secret is the right kind.  I.e. the user specifies `example-secret`, and the helm chart fails if the secret that exists is not the right kind... i.e. they accidentally supplied a basic-auth secret or something like that.  The ability to fail fast and provide a comprehensible error message is rather important.


On Fri, Jan 15, 2021 at 5:57 PM Matt Butcher <Matt.Butcher@...> wrote:
The preferred method is requiring the user to set a value that points to an existing secret. If the user points to the wrong thing, then Helm should fail to deploy that release.

We advise people not to write charts that try to scan for SSL certificates on clusters, as that is a security warning sign. So I would advise not using lookup​ in this (or any) case.

Helm is not and should not be used as a release manager tool. It is a package manager whose dependencies should be calculated up front and whose configuration should be passed by a user, not autoconfigured. In other words, think of Helm as 'apt-get', not 'chef' or 'puppet'.

From: cncf-helm@... <cncf-helm@...> on behalf of Loritsch, Berin via lists.cncf.io <bloritsch=novetta.com@...>
Sent: Friday, January 15, 2021 12:46 PM
To: cncf-helm@... <cncf-helm@...>
Subject: [EXTERNAL] Re: [cncf-helm] Is there any way to enforce constraints
 
It appears this did not garner any attention.  Is there any way to enforce constraints on a chart's dependencies?

For example, I would like to have my chart enforce a specific _type_ of secret the consumer provides.

My container can use TLS secrets and make use of them if they are provided.  However, it won't make sense if the user supplies a service account token or basic-auth token.  It would be fine if the secret is not provided at all, but if it is provided it should be a specific type.

I am clear on how to create the templates, and mount the secret to my container.  I'm not clear on how to enforce rather important constraints like that.

On Tue, Jan 12, 2021 at 10:17 AM Loritsch, Berin <bloritsch@...> wrote:
Example: I am creating a chart for a single page app server.  In it I want the ability to add a reference to the TLS secret.  The TLS secret would be provided by the person using the chart.

In my chart I want to enforce that the provided Secret is the type:  kubernetes.io/tls 

I want that constraint enforced prior to deploying the chart. 

--
Berin Loritsch

DOMEX Architect


7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708



--
Berin Loritsch

DOMEX Architect


7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708



--
Berin Loritsch

DOMEX Architect


7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708


Re: [EXTERNAL] Re: [cncf-helm] Is there any way to enforce constraints

Matt Butcher <matt.butcher@...>
 

The preferred method is requiring the user to set a value that points to an existing secret. If the user points to the wrong thing, then Helm should fail to deploy that release.

We advise people not to write charts that try to scan for SSL certificates on clusters, as that is a security warning sign. So I would advise not using lookup​ in this (or any) case.

Helm is not and should not be used as a release manager tool. It is a package manager whose dependencies should be calculated up front and whose configuration should be passed by a user, not autoconfigured. In other words, think of Helm as 'apt-get', not 'chef' or 'puppet'.


From: cncf-helm@... <cncf-helm@...> on behalf of Loritsch, Berin via lists.cncf.io <bloritsch=novetta.com@...>
Sent: Friday, January 15, 2021 12:46 PM
To: cncf-helm@... <cncf-helm@...>
Subject: [EXTERNAL] Re: [cncf-helm] Is there any way to enforce constraints
 
It appears this did not garner any attention.  Is there any way to enforce constraints on a chart's dependencies?

For example, I would like to have my chart enforce a specific _type_ of secret the consumer provides.

My container can use TLS secrets and make use of them if they are provided.  However, it won't make sense if the user supplies a service account token or basic-auth token.  It would be fine if the secret is not provided at all, but if it is provided it should be a specific type.

I am clear on how to create the templates, and mount the secret to my container.  I'm not clear on how to enforce rather important constraints like that.

On Tue, Jan 12, 2021 at 10:17 AM Loritsch, Berin <bloritsch@...> wrote:
Example: I am creating a chart for a single page app server.  In it I want the ability to add a reference to the TLS secret.  The TLS secret would be provided by the person using the chart.

In my chart I want to enforce that the provided Secret is the type:  kubernetes.io/tls 

I want that constraint enforced prior to deploying the chart. 

--
Berin Loritsch

DOMEX Architect


7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708



--
Berin Loritsch

DOMEX Architect


7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708


Re: Is there any way to enforce constraints

Loritsch, Berin <bloritsch@...>
 

Sorry, wrong email.


On Fri, Jan 15, 2021 at 4:46 PM Loritsch, Berin via lists.cncf.io <bloritsch=novetta.com@...> wrote:
Would 2pm work for you on monday?

On Fri, Jan 15, 2021 at 2:54 PM Paul Czarkowski <pczarkowski@...> wrote:

Hey Berin,

 

I’m not sure if this completely answers the question, but the helm lookup function (https://helm.sh/docs/chart_template_guide/functions_and_pipelines/)  may be usable here to introspect the secret and make sure it has the correct type or the correct keys in the data.

 

From: cncf-helm@... <cncf-helm@...> On Behalf Of Loritsch, Berin via lists.cncf.io
Sent: Friday, January 15, 2021 1:47 PM
To: cncf-helm@...
Subject: Re: [cncf-helm] Is there any way to enforce constraints

 

It appears this did not garner any attention.  Is there any way to enforce constraints on a chart's dependencies?

 

For example, I would like to have my chart enforce a specific _type_ of secret the consumer provides.

 

My container can use TLS secrets and make use of them if they are provided.  However, it won't make sense if the user supplies a service account token or basic-auth token.  It would be fine if the secret is not provided at all, but if it is provided it should be a specific type.

 

I am clear on how to create the templates, and mount the secret to my container.  I'm not clear on how to enforce rather important constraints like that.

 

On Tue, Jan 12, 2021 at 10:17 AM Loritsch, Berin <bloritsch@...> wrote:

Example: I am creating a chart for a single page app server.  In it I want the ability to add a reference to the TLS secret.  The TLS secret would be provided by the person using the chart.

 

In my chart I want to enforce that the provided Secret is the type:  kubernetes.io/tls 

 

I want that constraint enforced prior to deploying the chart. 

 

--

Berin Loritsch

DOMEX Architect

Image removed by sender.

7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708


 

--

Berin Loritsch

DOMEX Architect

Image removed by sender.

7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708



--
Berin Loritsch

DOMEX Architect


7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708



--
Berin Loritsch

DOMEX Architect


7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708


Re: Is there any way to enforce constraints

Loritsch, Berin <bloritsch@...>
 

Would 2pm work for you on monday?


On Fri, Jan 15, 2021 at 2:54 PM Paul Czarkowski <pczarkowski@...> wrote:

Hey Berin,

 

I’m not sure if this completely answers the question, but the helm lookup function (https://helm.sh/docs/chart_template_guide/functions_and_pipelines/)  may be usable here to introspect the secret and make sure it has the correct type or the correct keys in the data.

 

From: cncf-helm@... <cncf-helm@...> On Behalf Of Loritsch, Berin via lists.cncf.io
Sent: Friday, January 15, 2021 1:47 PM
To: cncf-helm@...
Subject: Re: [cncf-helm] Is there any way to enforce constraints

 

It appears this did not garner any attention.  Is there any way to enforce constraints on a chart's dependencies?

 

For example, I would like to have my chart enforce a specific _type_ of secret the consumer provides.

 

My container can use TLS secrets and make use of them if they are provided.  However, it won't make sense if the user supplies a service account token or basic-auth token.  It would be fine if the secret is not provided at all, but if it is provided it should be a specific type.

 

I am clear on how to create the templates, and mount the secret to my container.  I'm not clear on how to enforce rather important constraints like that.

 

On Tue, Jan 12, 2021 at 10:17 AM Loritsch, Berin <bloritsch@...> wrote:

Example: I am creating a chart for a single page app server.  In it I want the ability to add a reference to the TLS secret.  The TLS secret would be provided by the person using the chart.

 

In my chart I want to enforce that the provided Secret is the type:  kubernetes.io/tls 

 

I want that constraint enforced prior to deploying the chart. 

 

--

Berin Loritsch

DOMEX Architect

Image removed by sender.

7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708


 

--

Berin Loritsch

DOMEX Architect

Image removed by sender.

7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708



--
Berin Loritsch

DOMEX Architect


7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708


Re: Is there any way to enforce constraints

Paul Czarkowski
 

Hey Berin,

 

I’m not sure if this completely answers the question, but the helm lookup function (https://helm.sh/docs/chart_template_guide/functions_and_pipelines/)  may be usable here to introspect the secret and make sure it has the correct type or the correct keys in the data.

 

From: cncf-helm@... <cncf-helm@...> On Behalf Of Loritsch, Berin via lists.cncf.io
Sent: Friday, January 15, 2021 1:47 PM
To: cncf-helm@...
Subject: Re: [cncf-helm] Is there any way to enforce constraints

 

It appears this did not garner any attention.  Is there any way to enforce constraints on a chart's dependencies?

 

For example, I would like to have my chart enforce a specific _type_ of secret the consumer provides.

 

My container can use TLS secrets and make use of them if they are provided.  However, it won't make sense if the user supplies a service account token or basic-auth token.  It would be fine if the secret is not provided at all, but if it is provided it should be a specific type.

 

I am clear on how to create the templates, and mount the secret to my container.  I'm not clear on how to enforce rather important constraints like that.

 

On Tue, Jan 12, 2021 at 10:17 AM Loritsch, Berin <bloritsch@...> wrote:

Example: I am creating a chart for a single page app server.  In it I want the ability to add a reference to the TLS secret.  The TLS secret would be provided by the person using the chart.

 

In my chart I want to enforce that the provided Secret is the type:  kubernetes.io/tls 

 

I want that constraint enforced prior to deploying the chart. 

 

--

Berin Loritsch

DOMEX Architect

Image removed by sender.

7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708


 

--

Berin Loritsch

DOMEX Architect

Image removed by sender.

7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708


Re: Is there any way to enforce constraints

Loritsch, Berin <bloritsch@...>
 

It appears this did not garner any attention.  Is there any way to enforce constraints on a chart's dependencies?

For example, I would like to have my chart enforce a specific _type_ of secret the consumer provides.

My container can use TLS secrets and make use of them if they are provided.  However, it won't make sense if the user supplies a service account token or basic-auth token.  It would be fine if the secret is not provided at all, but if it is provided it should be a specific type.

I am clear on how to create the templates, and mount the secret to my container.  I'm not clear on how to enforce rather important constraints like that.

On Tue, Jan 12, 2021 at 10:17 AM Loritsch, Berin <bloritsch@...> wrote:
Example: I am creating a chart for a single page app server.  In it I want the ability to add a reference to the TLS secret.  The TLS secret would be provided by the person using the chart.

In my chart I want to enforce that the provided Secret is the type:  kubernetes.io/tls 

I want that constraint enforced prior to deploying the chart. 

--
Berin Loritsch

DOMEX Architect


7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708



--
Berin Loritsch

DOMEX Architect


7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708


Pre-proposal idea: introduce a new flag for helm uninstall "--fail-on-error" and a new release status "failed-uninstall"

mikeshngdev@...
 

Hi all,

Following the HIP guide. I am looking for feedback regarding my idea below:

The current Helm uninstall implementation is marking release record as uninstalled even if Helm detected errors during the uninstall process. See:

https://github.com/helm/helm/blob/v3.4.2/pkg/action/uninstall.go#L122-L127

https://github.com/helm/helm/blob/v3.4.2/pkg/action/uninstall.go#L144-L150


If the release record is marked as uninstalled then re-attempting to helm uninstall doesn't actually delete any resources (just purge the release record) . See:

https://github.com/helm/helm/blob/v3.4.2/pkg/action/uninstall.go#L81-L91

I think it's very useful for scripts or automation tools to be able to re-attempt a helm uninstall that resulted in error.

That's why I am sharing my idea of adding a new flag to helm uninstall "--fail-on-error".
If this flag is enabled, and the helm uninstall is successful then it doesn't change any behavior.

If this flag is enabled, and the helm uninstall completed with errors then update the release record to "failed-uninstall" status.

This allows the next helm uninstall attempt to actually perform resources delete.

For reference, I initially suggested https://github.com/helm/helm/issues/9201 but after thinking about it I think this "--fail-on-error" idea is much clearer and cleaner.

Thank you for reading and please provide some feedback. I am willing to drive the implementation if needed.

Mike Ng


Re: Proposed dates for 3.6 release

Marc Khouzam
 

There was no objections to the proposed dates for 3.6.  So with today's 3.5 release the 3.6 dates can be announced as:

3.6 RC1: Monday May 17th, 2021
3.6 final release: Wednesday May 26th, 2021

Thanks

On Thu, Jan 7, 2021 at 13:27 Marc Khouzam <marc.khouzam@...> wrote:
Hi,

helm 3.5 will be released next week Wednesday, January 13th as previously announced.
At the same time we should announce when the 3.6 release is planned.
On the dev call today it was suggested:

3.6 RC1: Monday May 17th, 2021
3.6 final release: Wednesday May 26th, 2021

That date is about a month following the release of Kubernetes 1.21 to give maintainers
enough time to adapt helm to any unexpected changes to Kubernetes's Go API.

Does anyone have concerns about those dates?  (there is a holiday very close following which may affect maintainers availability).

Let's do a lazy consensus, so if there are no objections by next Tuesday, we'll announce the above dates the next day.

Thank you

Marc



Is there any way to enforce constraints

Loritsch, Berin <bloritsch@...>
 

Example: I am creating a chart for a single page app server.  In it I want the ability to add a reference to the TLS secret.  The TLS secret would be provided by the person using the chart.

In my chart I want to enforce that the provided Secret is the type:  kubernetes.io/tls 

I want that constraint enforced prior to deploying the chart. 

--
Berin Loritsch

DOMEX Architect


7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708


Proposed dates for 3.6 release

Marc Khouzam
 

Hi,

helm 3.5 will be released next week Wednesday, January 13th as previously announced.
At the same time we should announce when the 3.6 release is planned.
On the dev call today it was suggested:

3.6 RC1: Monday May 17th, 2021
3.6 final release: Wednesday May 26th, 2021

That date is about a month following the release of Kubernetes 1.21 to give maintainers
enough time to adapt helm to any unexpected changes to Kubernetes's Go API.

Does anyone have concerns about those dates?  (there is a holiday very close following which may affect maintainers availability).

Let's do a lazy consensus, so if there are no objections by next Tuesday, we'll announce the above dates the next day.

Thank you

Marc



Re: Helm Test: What kind of tests can we perform on a helm chart?

Abhishek Tamrakar
 

Hi,
It depends what you are doing with the helm charts? Mostly people install apps, many install pre-requisites (roles, clusterroles, crds, etc.) apart from the applications (pods).
In case of application, a basic sanity of the application can be planned and could be written as part of tests/<template> in your helm chart wherever possible.
I don't think any other tests are necessary, because post deployment you can follow standard testing process which may include smoke tests, performance tests, penitration/security tests, etc.


Helm Test: What kind of tests can we perform on a helm chart?

@Tejaswini_Vadlamudi
 

Hi,
I was looking for the list of tests that can be executed or recommended to execute on a helm chart. Does anyone have a good handy list that can be seen as a good starting point?
One observation, most of the helm hub helm charts don't have tests. I'm not sure if they are late in implementing or if have any valid reason to NOT deliver tests.

Thanks!
Teja


Helm v3.4, Helm v2.17, and Stable/Incubator Updates

Matt Farina
 

Helm v3.4.0 and v2.17.0 were both released. Along with that the details on the new location for the stable and incubator chart repositories are available. Details are in the blog post at https://helm.sh/blog/new-location-stable-incubator-charts/


Helm Feature Proposal: Fail-Faster Testing

黃健毅 <johnny.jy.ooi@...>
 

Good morning/afternoon/evening (delete as applicable)

I've been directed to this mailing list by Martin Hickey and Matt Fisher due to a proposal I have briefly mentioned at https://github.com/helm/helm/issues/8861

I will put more details in this email

_____

Background

When deploying a Helm chart, you normally need to wait for the resources to be deployed by Helm, then use something like "kubectl rollout deployments/{deploymentname}" to wait for the rollout to finish before running "helm test" on the chart -- as if you ran the Helm test before the rollout was completed, and your test accesses the service via the cluster service (ClusterIP/NodePort/LoadBalancer), you would get a split between the old and new versions of the chart.

And herein lies the problem. If you have a chart that might have any number of features such as:

1) has a large number of replicas
2) has a slow readiness process (e.g. due to cluster-like syncing behaviour such as ElasticSearch)

Then the rollout state will take a long time -- during which, you can't run "helm test" because of the "split brain" situation and potentially inconsistent results.

If you wait until the rollout finishes, then run the test, and find the test has failed (e.g. for a complete breakage with the update), you then need to roll back the deployment -- this then takes the same amount of time to rolling update the pods back to the previous version as it did to update them to the new version. And when you have spent 30 minutes updating the pods, the last thing you want to have to do is have to do it again.

However, from the end user/customer perspective, the service will be down 100% in between the broken update being applied, and the rollback succeeding (if it does succeed)

____

Example

Let's assume we have a four-pod service based on nginx, with our website baked into the custom image used by the pods, represented here as "O"s

OOOO

We apply the update, and the deployment has a surge of two pods, so two additional pods come up -- we'll represent these as "n"s

OOOOnn

The two new pods come good -- represented as "N"s, and Kubernetes terminates two existing pods -- represented as "X"s. At this point, if the update was bad, we now have 50% of traffic going to the two old pods (and being successful), and 50% going to the new pods (and failing)

OOXXNN

Kubernetes continues on. The terminated pods disappear, and it spins up two new pods

OONNnn

The new pods come good and it terminates the final two old pods

XXNNNN

NNNN

We now have four new pods, but at this state, 100% of the traffic going through this service is failing. We run "helm test" which fails, and we rollback back to the good state

____

Proposal

The proposal of this change is to allow (opt-in) the testing to be applied earlier, preferably during the pod rollout, so that if the test fails, the rollout is cancelled while it is still in progress, reducing the downtime impact and minimising the rollback time. Using the example above, the flow would look like this:

4 pods as before:

OOOO

2 new pods are created

OOOOnn

Those two pods come good, and Kubernetes terminates two existing pods

OOXXNN

Helm detects the new pods are now good, and actions tests on one of them (or both, if the "--parallel" switch is used) -- the pod being testing is marked with T

OOTN

Test fails on the pod. So Helm cancels the rollout and triggers a rollback of the changes.

At this precise moment in time, we have two old pods and two new pods, so traffic is split 50/50 between the pods. So instead of your service being broken for 100% of traffic, your service is broken for 50% and rollback only requires creation to two pods (with the old configuration), allowing a faster rollback and reduced downtime.

If we had 100 replicas instead of four, we could find a broken update 2% into the update (assuming a surge of 2 pods), rather than after 100% into the update, and save the time of having to roll back 100 pods,

____


Happy to respond to any questions or suggestions to this proposal.

Regards

Johnny Ooi



Helm's 5th Birthday

Matt Farina
 

Today, Oct 19th, is Helm's 5th birthday. Along with that we have an announcement about GitHub and the stable/incubator charts... https://helm.sh/blog/helm-turns-five/


Helm Hub Migrated To Artifact Hub

Matt Farina
 

The Helm Hub has now migrated to the Artifact Hub. Details are available in the blog announcement at https://helm.sh/blog/helm-hub-moving-to-artifact-hub/

If there are any questions or comments there are some details in the announcement on the best places to share those. Or, you can respond here and I'll try to help.

- Matt Farina


Re: Why do we need Helm Tests? What does they solve?

Loritsch, Berin <bloritsch@...>
 

My understanding is that Terratest was designed around testing deployments and predates any testing code in Helm.  I'm not on the terratest email distribution list so I don't know if they are even aware of helm's new testing support.


On Thu, Oct 1, 2020 at 6:12 AM <vtejaswini1@...> wrote:
@Matt:  I am trying to understand different test cases that can be tested via the helm test. I now agree that we need these tests but I was unsure how can help/aid my chart consumers. Most of the helm charts today in helm hub are testing connections to their ports. I was thinking if we can use "helm tests" to notice if the deployment was successful or not i.e. to understand if microservices are deployed properly as per deployment configuration and ready to serve traffic. Am I thinking correctly? Can we use helm tests to test readiness and liveness checks of microservices or is it a bad idea (rely on monitoring instead)?

@Berin: Thanks for the suggestions. I had a quick look at Terratest. If we have helm tests, we can either run "helm test" or "ct lint". I think Terratest is a go library support for writing test code and also offers support for deployment, validation of charts, and infrastructure readiness. Does Terratest run "helm test" as part of its validation or does that something else?

Thanks,
Teja



--
Berin Loritsch

Systems Integration Lead


7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708

101 - 120 of 457