Date   

October 20 TOC meeting cancelled

Amye Scavarda Perrin
 

We will not be meeting tomorrow, please take this time and record your KubeCon + Cloud Native Con virtual talks! 

(Suggested use of time.)
- amye 

--
Amye Scavarda Perrin | Program Manager | amye@...


[cncf-sig-security] Cloud Native Security Whitepaper is open for review and comment

Chris Aniszczyk
 

FYI this is great work and should get greater input

---------- Forwarded message ---------
From: Emily Fox <themoxiefoxatwork@...>
Date: Mon, Oct 19, 2020 at 3:59 PM
Subject: [cncf-sig-security] Cloud Native Security Whitepaper is open for review and comment
To: <cncf-sig-security@...>


Hello fellow security fans!
  The cloud native security whitepaper working group has done an excellent job pulling together some fabulous content and we need your help! Please take some time to review and comment on our paper.  We have one area still awaiting input from the great folks in SIG-Storage, but otherwise we welcome your review! 

CNSWP for review
Please have all comments and review complete by October 27th.  We appreciate your time and your help!

~Emily Fox,  co-chair SIG-Security



--
Chris Aniszczyk (@cra)


Re: [VOTE] Open Policy Agent from incubating to graduated

Justin Cappos <jcappos@...>
 

+1 NB

On Sat, Oct 17, 2020 at 7:23 AM Emily Fox <themoxiefoxatwork@...> wrote:

+1NB
- Emily Fox

@TheMoxieFox (personal handle)

On Fri, 16 Oct 2020, 19:19 John Hillegass, <hillegassdev@...> wrote:
+1 non-binding 
On Oct 16, 2020, 6:44 PM -0400, Chris Short via lists.cncf.io <chris=chrisshort.net@...>, wrote:
+1 non-binding 

On Thu, Oct 8, 2020 at 22:10 Archy k <ayrat.khayretdinov@...> wrote:
+1 NB

On Thu, Oct 8, 2020 at 5:46 PM Jon Mittelhauser <jon.mittelhauser@...> wrote:

+1 nb

 

From: <cncf-toc@...> on behalf of "Sheng Liang via lists.cncf.io" <sheng=rancher.com@...>
Reply-To: <sheng@...>
Date: Thursday, October 8, 2020 at 2:00 PM
To: "aprokharchyk@..." <aprokharchyk@...>, Amye Scavarda Perrin <ascavarda@...>
Cc: CNCF TOC <cncf-toc@...>
Subject: Re: [cncf-toc] [VOTE] Open Policy Agent from incubating to graduated

 

+1 binding

 

From: <cncf-toc@...> on behalf of "Alena Prokharchyk via lists.cncf.io" <aprokharchyk=apple.com@...>
Reply-To: "aprokharchyk@..." <aprokharchyk@...>
Date: Thursday, October 8, 2020 at 1:25 PM
To: Amye Scavarda Perrin <ascavarda@...>
Cc: CNCF TOC <cncf-toc@...>
Subject: Re: [cncf-toc] [VOTE] Open Policy Agent from incubating to graduated

 

+1 binding

 

-alena.




On Sep 30, 2020, at 9:00 AM, Amye Scavarda Perrin <ascavarda@...> wrote:

 

The Open Policy Agent project has applied for graduation from incubation to graduated. (https://github.com/cncf/toc/pull/520)

The due diligence document can be found here: https://docs.google.com/document/d/19M5fTpe57rQIMNxawRl5wSWvJUapuzY-CkV4O5pvieU/edit
 
Brendan Burns has called for public comment: https://lists.cncf.io/g/cncf-toc/message/5281

Please vote (+1/0/-1) by replying to this thread.

Remember that the TOC has binding votes only, but we do appreciate non-binding votes from the community as a sign of support!

 

--

Amye Scavarda Perrin | Program Manager | amye@...

 

--

Chris Short
He/Him/His


Re: [VOTE] Open Policy Agent from incubating to graduated

Emily Fox
 


+1NB
- Emily Fox

@TheMoxieFox (personal handle)

On Fri, 16 Oct 2020, 19:19 John Hillegass, <hillegassdev@...> wrote:
+1 non-binding 
On Oct 16, 2020, 6:44 PM -0400, Chris Short via lists.cncf.io <chris=chrisshort.net@...>, wrote:
+1 non-binding 

On Thu, Oct 8, 2020 at 22:10 Archy k <ayrat.khayretdinov@...> wrote:
+1 NB

On Thu, Oct 8, 2020 at 5:46 PM Jon Mittelhauser <jon.mittelhauser@...> wrote:

+1 nb

 

From: <cncf-toc@...> on behalf of "Sheng Liang via lists.cncf.io" <sheng=rancher.com@...>
Reply-To: <sheng@...>
Date: Thursday, October 8, 2020 at 2:00 PM
To: "aprokharchyk@..." <aprokharchyk@...>, Amye Scavarda Perrin <ascavarda@...>
Cc: CNCF TOC <cncf-toc@...>
Subject: Re: [cncf-toc] [VOTE] Open Policy Agent from incubating to graduated

 

+1 binding

 

From: <cncf-toc@...> on behalf of "Alena Prokharchyk via lists.cncf.io" <aprokharchyk=apple.com@...>
Reply-To: "aprokharchyk@..." <aprokharchyk@...>
Date: Thursday, October 8, 2020 at 1:25 PM
To: Amye Scavarda Perrin <ascavarda@...>
Cc: CNCF TOC <cncf-toc@...>
Subject: Re: [cncf-toc] [VOTE] Open Policy Agent from incubating to graduated

 

+1 binding

 

-alena.




On Sep 30, 2020, at 9:00 AM, Amye Scavarda Perrin <ascavarda@...> wrote:

 

The Open Policy Agent project has applied for graduation from incubation to graduated. (https://github.com/cncf/toc/pull/520)

The due diligence document can be found here: https://docs.google.com/document/d/19M5fTpe57rQIMNxawRl5wSWvJUapuzY-CkV4O5pvieU/edit
 
Brendan Burns has called for public comment: https://lists.cncf.io/g/cncf-toc/message/5281

Please vote (+1/0/-1) by replying to this thread.

Remember that the TOC has binding votes only, but we do appreciate non-binding votes from the community as a sign of support!

 

--

Amye Scavarda Perrin | Program Manager | amye@...

 

--

Chris Short
He/Him/His


Re: [VOTE] Open Policy Agent from incubating to graduated

John Hillegass
 

+1 non-binding 
On Oct 16, 2020, 6:44 PM -0400, Chris Short via lists.cncf.io <chris=chrisshort.net@...>, wrote:

+1 non-binding 

On Thu, Oct 8, 2020 at 22:10 Archy k <ayrat.khayretdinov@...> wrote:
+1 NB

On Thu, Oct 8, 2020 at 5:46 PM Jon Mittelhauser <jon.mittelhauser@...> wrote:

+1 nb

 

From: <cncf-toc@...> on behalf of "Sheng Liang via lists.cncf.io" <sheng=rancher.com@...>
Reply-To: <sheng@...>
Date: Thursday, October 8, 2020 at 2:00 PM
To: "aprokharchyk@..." <aprokharchyk@...>, Amye Scavarda Perrin <ascavarda@...>
Cc: CNCF TOC <cncf-toc@...>
Subject: Re: [cncf-toc] [VOTE] Open Policy Agent from incubating to graduated

 

+1 binding

 

From: <cncf-toc@...> on behalf of "Alena Prokharchyk via lists.cncf.io" <aprokharchyk=apple.com@...>
Reply-To: "aprokharchyk@..." <aprokharchyk@...>
Date: Thursday, October 8, 2020 at 1:25 PM
To: Amye Scavarda Perrin <ascavarda@...>
Cc: CNCF TOC <cncf-toc@...>
Subject: Re: [cncf-toc] [VOTE] Open Policy Agent from incubating to graduated

 

+1 binding

 

-alena.




On Sep 30, 2020, at 9:00 AM, Amye Scavarda Perrin <ascavarda@...> wrote:

 

The Open Policy Agent project has applied for graduation from incubation to graduated. (https://github.com/cncf/toc/pull/520)

The due diligence document can be found here: https://docs.google.com/document/d/19M5fTpe57rQIMNxawRl5wSWvJUapuzY-CkV4O5pvieU/edit
 
Brendan Burns has called for public comment: https://lists.cncf.io/g/cncf-toc/message/5281

Please vote (+1/0/-1) by replying to this thread.

Remember that the TOC has binding votes only, but we do appreciate non-binding votes from the community as a sign of support!

 

--

Amye Scavarda Perrin | Program Manager | amye@...

 

--

Chris Short
He/Him/His


Re: [VOTE] Open Policy Agent from incubating to graduated

Chris Short
 

+1 non-binding 

On Thu, Oct 8, 2020 at 22:10 Archy k <ayrat.khayretdinov@...> wrote:
+1 NB

On Thu, Oct 8, 2020 at 5:46 PM Jon Mittelhauser <jon.mittelhauser@...> wrote:

+1 nb

 

From: <cncf-toc@...> on behalf of "Sheng Liang via lists.cncf.io" <sheng=rancher.com@...>
Reply-To: <sheng@...>
Date: Thursday, October 8, 2020 at 2:00 PM
To: "aprokharchyk@..." <aprokharchyk@...>, Amye Scavarda Perrin <ascavarda@...>
Cc: CNCF TOC <cncf-toc@...>
Subject: Re: [cncf-toc] [VOTE] Open Policy Agent from incubating to graduated

 

+1 binding

 

From: <cncf-toc@...> on behalf of "Alena Prokharchyk via lists.cncf.io" <aprokharchyk=apple.com@...>
Reply-To: "aprokharchyk@..." <aprokharchyk@...>
Date: Thursday, October 8, 2020 at 1:25 PM
To: Amye Scavarda Perrin <ascavarda@...>
Cc: CNCF TOC <cncf-toc@...>
Subject: Re: [cncf-toc] [VOTE] Open Policy Agent from incubating to graduated

 

+1 binding

 

-alena.




On Sep 30, 2020, at 9:00 AM, Amye Scavarda Perrin <ascavarda@...> wrote:

 

The Open Policy Agent project has applied for graduation from incubation to graduated. (https://github.com/cncf/toc/pull/520)

The due diligence document can be found here: https://docs.google.com/document/d/19M5fTpe57rQIMNxawRl5wSWvJUapuzY-CkV4O5pvieU/edit
 
Brendan Burns has called for public comment: https://lists.cncf.io/g/cncf-toc/message/5281

Please vote (+1/0/-1) by replying to this thread.

Remember that the TOC has binding votes only, but we do appreciate non-binding votes from the community as a sign of support!

 

--

Amye Scavarda Perrin | Program Manager | amye@...

 

--

Chris Short
He/Him/His


[RFC] Discussing/modifying the kubernetes/kubernetes minor release cadence

Stephen Augustus
 

Forwarding this along to the TOC and SIG Contributor Strategy, as it has project implications for downstream consumers of Kubernetes.

-- Stephen

---------- Forwarded message ---------
From: Stephen Augustus <stephen.k8s@...>
Date: Thu, Oct 15, 2020 at 12:09 PM
Subject: [kubernetes-sig-testing] [RFC] Discussing/modifying the kubernetes/kubernetes minor release cadence
To: Kubernetes developer/contributor discussion <kubernetes-dev@...>
Cc: kubernetes-sig-release <kubernetes-sig-release@...>, <sig-release-leads@...>, kubernetes-sig-architecture <kubernetes-sig-architecture@...>, kubernetes-sig-testing <kubernetes-sig-testing@...>, kubernetes-sig-cluster-lifecycle <kubernetes-sig-cluster-lifecycle@...>


Hey Kubernetes Community!

Following the extended 1.19 cycle, we've received several questions across a variety of platforms and DMs about whether the project is intending to only have three minor releases/year moving forward.

So let's discuss this together and see if it makes sense to make some changes going into the new year!

If you are in any way invested in how often we cut Kubernetes minor releases, please provide some feedback on the following issue: https://github.com/kubernetes/sig-release/issues/1290

I'm keeping this email intentionally light, because I'd like to capture responses on the issue, instead of this thread.

I'll note this in today's Community meeting and would request/encourage you to raise this topic in your respective governance groups, as this affects us all.

Looking forward to the discussion!
Stephen

--
You received this message because you are subscribed to the Google Groups "kubernetes-sig-testing" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-testing+unsubscribe@....
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-testing/CAOqU-DQ%2BwdsQA%3D%3DH28h1qomq8jBNUiVum_ObGHbHZQKinFyqyA%40mail.gmail.com.


Re: [URGENT] Impending deletion of Helm 2 releases and charts buckets

Vic Iglesias <viglesias@...>
 

I had 2 suggestions wrt mucking with index.yaml:

  1. Use index.yaml to point all chart downloads to a specially crafted chart that causes a hard failure on the users end but provides an error message that points them at remediation steps.

    1. Pros

      1. Gives existing users an error message that helps them with the migration instead of a 404 with no guidance.

    2. Cons

      1. Needs to be tested thoroughly. We don't want Helm to delete any user resources when it encounters this Chart. It must be an early failure with no side effects.

      2. Still some bandwidth costs even if the chart is very small.

  2. Use the index.yaml to redirect all chart downloads to Chart Center.

    1. Pros

      1. Easy change for maintainers

      2. No change for users

    2. Cons

      1. Requires JFrog to foot the bandwidth bill for chart tarballs

      2. Requires Google to foot the bill for index.yaml

      3. Creates a crutch for users on the deprecated versions of Helm that will eventually need to be removed as well. We're kinda just kicking the can down the road.


On Wed, Oct 14, 2020 at 1:24 PM Josh Dolitsky <jdolitsky@...> wrote:
Brian/Vic - are you saying continue serving index.yaml from GCS, but modify it to point to .tgz files hosted elsewhere?

such as:

entries:
  alpine:
    - created: 2016-10-06T16:23:20.499814565-06:00
      description: Deploy a basic Alpine Linux pod
      digest: 99c76e403d752c84ead610644d4b1c2f2b453a74b921f422b9dcb8a7c8b559cd
      name: alpine
      urls:
      - https://otherdomain.com/charts/alpine-0.2.0.tgz
      version: 0.2.0

?

This might cut down on transfer costs, but I think the index is actually larger in size than the .tgz's.

Is that what you mean by "redirect via index files"?

- Josh

On Wed, Oct 14, 2020 at 3:48 PM Brian Grant via lists.cncf.io <briangrant=google.com@...> wrote:

On Wed, Oct 14, 2020 at 12:27 PM Matt Farina <matt@...> wrote:

From what I have been told, it's not technically possible to transfer the project.

That's not correct. Who told you that? As I pointed out on https://github.com/helm/community/issues/114:

Transferring the GCP Project to a CNCF GCP Organization and Account would make it easier for you to manage, and has to be done if it's going to continue to exist.

Sorry for the confusion. I've asked multiple people at Google about this and I was told it wasn't possible. Thank you for not only clearing up the confusion but pointing out the project-migration docs.

This still doesn't deal with the funding issue. We've not gone to the GB on funding (it would be well over the limit to just do so GB is needed) because the amount would be high enough that it impacts other CNCF activities like security audits and contractors working on things. We don't want to see that happen.

Now, if Helm could use some of the unused GCP credits for k8s the funding wouldn't be a problem. There are more than enough unused credits. Is transferring the bucket to the CNCF k8s account and setting it up to use some of those credits possible? We would shrink down what the buckets served and it would enable the k8s SIG Service catalog Helm repo to stay alive as it's managed through the same project.

You, Michelle Noorali, and Adam Reese have "Storage Admin" access. There's also a chart uploader service account.

This highlights part of the problem. If we look at the current Helm governance this isn't a list of the right people. Sadly, my access didn't even let me see who the other people were or that my access was "Storage Admin". IAM is a finicky thing.

The lack of ownership is more than a storage problem. For example, not having access to setup a custom domain with HTTPS. You need more than storage admin for that. This is what I meant by a pain point. Or, being able to manage who has access. Of the charts maintainers, I'm the only one on that list. Adam and Michelle aren't. What if I'm unavailable and someone else needs to step in? The reality is that "storage admin" isn't enough to really manage something well. That's what I meant.

Right. There are Googlers who can grant additional permissions, but moving the Project would fix this completely.

Vic had another suggestion: is it possible to redirect via Helm index files?
 

- Matt Farina

On Wed, Oct 14, 2020, at 2:53 PM, Brian Grant wrote:
On Wed, Oct 14, 2020 at 11:50 AM Matt Farina <matt@...> wrote:

From what I have been told, it's not technically possible to transfer the project.

That's not correct. Who told you that? As I pointed out on https://github.com/helm/community/issues/114:

Transferring the GCP Project to a CNCF GCP Organization and Account would make it easier for you to manage, and has to be done if it's going to continue to exist.


You, Michelle Noorali, and Adam Reese have "Storage Admin" access. There's also a chart uploader service account.

 


To add some context about cost and history...

We have seen numbers on what it would cost to host these through Google Cloud. The numbers, if I understood what I saw right, is the public cost rather than the internal cross charge cost. It's a larger number than I ever expected and a cost I'm very hesitant to ask the CNCF to cover as an unexpected expense. Especially during these times.

If there were a bunch of unused Google Cloud Credits laying around I would be happy to use them. But, that doesn't appear to be the case.

Just know, the cost to the CNCF would be substantial.

One of the problems is the hard coded URLs in the Helm client. This was all setup in early 2016 when Jack Greenfield and Ville Aikas were major Helm contributors on behalf of Google. The bucket setup (as with all the Helm Google Cloud services) were setup through Google corp accounts. Googlers controlled them and do to this day.

The lack of ownership by the project has been a pain point. But, older Helm v2 clients are stuck with it (can't do a redirect) and many of them are still in use. The version use across Helm versions is rather large.

What we want is to deprecate the charts repository (headed toward a distributed route) while still keeping things available for people. That includes old versions of charts.

I hope that context helps.

- Matt Farina

On Wed, Oct 14, 2020, at 2:10 PM, Josh Berkus wrote:
On 10/14/20 10:52 AM, Matt Farina wrote:

> There are two things worth noting...

>  1. Many of the charts people use have been slower to move out of the
>     stable and incubator repos than expected. We are still finding them
>     new homes. So, not everything is deprecated. There are many useful
>     things there.
>  2. There is more use of the stable and incubator repos than I ever
>     expected. I'm told the use is growing even as things move off it.
>     When I recently saw some numbers I was amazingly surprised. I had
>     expected it to be lower. Note, I don't have access to this data. The
>     buckets are Google owned/managed so only they have access to this. I
>     think Vic is the only one of the Helm maintainers with access to this.

> Many people will have some pain when the stable and incubator repos go away.

My question is: is the *Helm project* asking for the CNCF to fund
extended storage for these charts, or not? (assuming this is technically
possible).  I can see why the answer might be "not"; we have break the
links to the old locations sometime, and why not now?

The request to the TOC came from a concerned contributor, not from the
project leadership, as far as I can tell.

For my part, I would not want to see the TOC messing with a project's
official deprecation plans without a formal project request.

-- 
--
Josh Berkus
Kubernetes Community
Red Hat OSPO







Re: [URGENT] Impending deletion of Helm 2 releases and charts buckets

Josh Dolitsky
 

Brian/Vic - are you saying continue serving index.yaml from GCS, but modify it to point to .tgz files hosted elsewhere?

such as:

entries:
  alpine:
    - created: 2016-10-06T16:23:20.499814565-06:00
      description: Deploy a basic Alpine Linux pod
      digest: 99c76e403d752c84ead610644d4b1c2f2b453a74b921f422b9dcb8a7c8b559cd
      name: alpine
      urls:
      - https://otherdomain.com/charts/alpine-0.2.0.tgz
      version: 0.2.0

?

This might cut down on transfer costs, but I think the index is actually larger in size than the .tgz's.

Is that what you mean by "redirect via index files"?

- Josh

On Wed, Oct 14, 2020 at 3:48 PM Brian Grant via lists.cncf.io <briangrant=google.com@...> wrote:

On Wed, Oct 14, 2020 at 12:27 PM Matt Farina <matt@...> wrote:

From what I have been told, it's not technically possible to transfer the project.

That's not correct. Who told you that? As I pointed out on https://github.com/helm/community/issues/114:

Transferring the GCP Project to a CNCF GCP Organization and Account would make it easier for you to manage, and has to be done if it's going to continue to exist.

Sorry for the confusion. I've asked multiple people at Google about this and I was told it wasn't possible. Thank you for not only clearing up the confusion but pointing out the project-migration docs.

This still doesn't deal with the funding issue. We've not gone to the GB on funding (it would be well over the limit to just do so GB is needed) because the amount would be high enough that it impacts other CNCF activities like security audits and contractors working on things. We don't want to see that happen.

Now, if Helm could use some of the unused GCP credits for k8s the funding wouldn't be a problem. There are more than enough unused credits. Is transferring the bucket to the CNCF k8s account and setting it up to use some of those credits possible? We would shrink down what the buckets served and it would enable the k8s SIG Service catalog Helm repo to stay alive as it's managed through the same project.

You, Michelle Noorali, and Adam Reese have "Storage Admin" access. There's also a chart uploader service account.

This highlights part of the problem. If we look at the current Helm governance this isn't a list of the right people. Sadly, my access didn't even let me see who the other people were or that my access was "Storage Admin". IAM is a finicky thing.

The lack of ownership is more than a storage problem. For example, not having access to setup a custom domain with HTTPS. You need more than storage admin for that. This is what I meant by a pain point. Or, being able to manage who has access. Of the charts maintainers, I'm the only one on that list. Adam and Michelle aren't. What if I'm unavailable and someone else needs to step in? The reality is that "storage admin" isn't enough to really manage something well. That's what I meant.

Right. There are Googlers who can grant additional permissions, but moving the Project would fix this completely.

Vic had another suggestion: is it possible to redirect via Helm index files?
 

- Matt Farina

On Wed, Oct 14, 2020, at 2:53 PM, Brian Grant wrote:
On Wed, Oct 14, 2020 at 11:50 AM Matt Farina <matt@...> wrote:

From what I have been told, it's not technically possible to transfer the project.

That's not correct. Who told you that? As I pointed out on https://github.com/helm/community/issues/114:

Transferring the GCP Project to a CNCF GCP Organization and Account would make it easier for you to manage, and has to be done if it's going to continue to exist.


You, Michelle Noorali, and Adam Reese have "Storage Admin" access. There's also a chart uploader service account.

 


To add some context about cost and history...

We have seen numbers on what it would cost to host these through Google Cloud. The numbers, if I understood what I saw right, is the public cost rather than the internal cross charge cost. It's a larger number than I ever expected and a cost I'm very hesitant to ask the CNCF to cover as an unexpected expense. Especially during these times.

If there were a bunch of unused Google Cloud Credits laying around I would be happy to use them. But, that doesn't appear to be the case.

Just know, the cost to the CNCF would be substantial.

One of the problems is the hard coded URLs in the Helm client. This was all setup in early 2016 when Jack Greenfield and Ville Aikas were major Helm contributors on behalf of Google. The bucket setup (as with all the Helm Google Cloud services) were setup through Google corp accounts. Googlers controlled them and do to this day.

The lack of ownership by the project has been a pain point. But, older Helm v2 clients are stuck with it (can't do a redirect) and many of them are still in use. The version use across Helm versions is rather large.

What we want is to deprecate the charts repository (headed toward a distributed route) while still keeping things available for people. That includes old versions of charts.

I hope that context helps.

- Matt Farina

On Wed, Oct 14, 2020, at 2:10 PM, Josh Berkus wrote:
On 10/14/20 10:52 AM, Matt Farina wrote:

> There are two things worth noting...

>  1. Many of the charts people use have been slower to move out of the
>     stable and incubator repos than expected. We are still finding them
>     new homes. So, not everything is deprecated. There are many useful
>     things there.
>  2. There is more use of the stable and incubator repos than I ever
>     expected. I'm told the use is growing even as things move off it.
>     When I recently saw some numbers I was amazingly surprised. I had
>     expected it to be lower. Note, I don't have access to this data. The
>     buckets are Google owned/managed so only they have access to this. I
>     think Vic is the only one of the Helm maintainers with access to this.

> Many people will have some pain when the stable and incubator repos go away.

My question is: is the *Helm project* asking for the CNCF to fund
extended storage for these charts, or not? (assuming this is technically
possible).  I can see why the answer might be "not"; we have break the
links to the old locations sometime, and why not now?

The request to the TOC came from a concerned contributor, not from the
project leadership, as far as I can tell.

For my part, I would not want to see the TOC messing with a project's
official deprecation plans without a formal project request.

-- 
--
Josh Berkus
Kubernetes Community
Red Hat OSPO







Re: [URGENT] Impending deletion of Helm 2 releases and charts buckets

Brian Grant
 


On Wed, Oct 14, 2020 at 12:27 PM Matt Farina <matt@...> wrote:

From what I have been told, it's not technically possible to transfer the project.

That's not correct. Who told you that? As I pointed out on https://github.com/helm/community/issues/114:

Transferring the GCP Project to a CNCF GCP Organization and Account would make it easier for you to manage, and has to be done if it's going to continue to exist.

Sorry for the confusion. I've asked multiple people at Google about this and I was told it wasn't possible. Thank you for not only clearing up the confusion but pointing out the project-migration docs.

This still doesn't deal with the funding issue. We've not gone to the GB on funding (it would be well over the limit to just do so GB is needed) because the amount would be high enough that it impacts other CNCF activities like security audits and contractors working on things. We don't want to see that happen.

Now, if Helm could use some of the unused GCP credits for k8s the funding wouldn't be a problem. There are more than enough unused credits. Is transferring the bucket to the CNCF k8s account and setting it up to use some of those credits possible? We would shrink down what the buckets served and it would enable the k8s SIG Service catalog Helm repo to stay alive as it's managed through the same project.

You, Michelle Noorali, and Adam Reese have "Storage Admin" access. There's also a chart uploader service account.

This highlights part of the problem. If we look at the current Helm governance this isn't a list of the right people. Sadly, my access didn't even let me see who the other people were or that my access was "Storage Admin". IAM is a finicky thing.

The lack of ownership is more than a storage problem. For example, not having access to setup a custom domain with HTTPS. You need more than storage admin for that. This is what I meant by a pain point. Or, being able to manage who has access. Of the charts maintainers, I'm the only one on that list. Adam and Michelle aren't. What if I'm unavailable and someone else needs to step in? The reality is that "storage admin" isn't enough to really manage something well. That's what I meant.

Right. There are Googlers who can grant additional permissions, but moving the Project would fix this completely.

Vic had another suggestion: is it possible to redirect via Helm index files?
 

- Matt Farina

On Wed, Oct 14, 2020, at 2:53 PM, Brian Grant wrote:
On Wed, Oct 14, 2020 at 11:50 AM Matt Farina <matt@...> wrote:

From what I have been told, it's not technically possible to transfer the project.

That's not correct. Who told you that? As I pointed out on https://github.com/helm/community/issues/114:

Transferring the GCP Project to a CNCF GCP Organization and Account would make it easier for you to manage, and has to be done if it's going to continue to exist.


You, Michelle Noorali, and Adam Reese have "Storage Admin" access. There's also a chart uploader service account.

 


To add some context about cost and history...

We have seen numbers on what it would cost to host these through Google Cloud. The numbers, if I understood what I saw right, is the public cost rather than the internal cross charge cost. It's a larger number than I ever expected and a cost I'm very hesitant to ask the CNCF to cover as an unexpected expense. Especially during these times.

If there were a bunch of unused Google Cloud Credits laying around I would be happy to use them. But, that doesn't appear to be the case.

Just know, the cost to the CNCF would be substantial.

One of the problems is the hard coded URLs in the Helm client. This was all setup in early 2016 when Jack Greenfield and Ville Aikas were major Helm contributors on behalf of Google. The bucket setup (as with all the Helm Google Cloud services) were setup through Google corp accounts. Googlers controlled them and do to this day.

The lack of ownership by the project has been a pain point. But, older Helm v2 clients are stuck with it (can't do a redirect) and many of them are still in use. The version use across Helm versions is rather large.

What we want is to deprecate the charts repository (headed toward a distributed route) while still keeping things available for people. That includes old versions of charts.

I hope that context helps.

- Matt Farina

On Wed, Oct 14, 2020, at 2:10 PM, Josh Berkus wrote:
On 10/14/20 10:52 AM, Matt Farina wrote:

> There are two things worth noting...

>  1. Many of the charts people use have been slower to move out of the
>     stable and incubator repos than expected. We are still finding them
>     new homes. So, not everything is deprecated. There are many useful
>     things there.
>  2. There is more use of the stable and incubator repos than I ever
>     expected. I'm told the use is growing even as things move off it.
>     When I recently saw some numbers I was amazingly surprised. I had
>     expected it to be lower. Note, I don't have access to this data. The
>     buckets are Google owned/managed so only they have access to this. I
>     think Vic is the only one of the Helm maintainers with access to this.

> Many people will have some pain when the stable and incubator repos go away.

My question is: is the *Helm project* asking for the CNCF to fund
extended storage for these charts, or not? (assuming this is technically
possible).  I can see why the answer might be "not"; we have break the
links to the old locations sometime, and why not now?

The request to the TOC came from a concerned contributor, not from the
project leadership, as far as I can tell.

For my part, I would not want to see the TOC messing with a project's
official deprecation plans without a formal project request.

-- 
--
Josh Berkus
Kubernetes Community
Red Hat OSPO







Re: [URGENT] Impending deletion of Helm 2 releases and charts buckets

Liz Rice
 

For my part, I would not want to see the TOC messing with a project's
official deprecation plans without a formal project request.
To be clear, what I was suggesting was not intended to “mess with anyone’s plans” - I was simply trying to lay out that if Helm finds itself in a difficult position here, we (TOC) are here to help, be that lobbying for additional support in marketing or even budget, if that’s appropriate. Helm is a graduated project; that’s supposed to mean minimal risk for end users of the project; a risk has been identified and I think it is entirely appropriate for TOC to offer help & support in an attempt to mitigate that risk. 



On Wed, 14 Oct 2020 at 19:53, Brian Grant via lists.cncf.io <briangrant=google.com@...> wrote:
On Wed, Oct 14, 2020 at 11:50 AM Matt Farina <matt@...> wrote:
From what I have been told, it's not technically possible to transfer the project.

That's not correct. Who told you that? As I pointed out on https://github.com/helm/community/issues/114:

Transferring the GCP Project to a CNCF GCP Organization and Account would make it easier for you to manage, and has to be done if it's going to continue to exist.

You, Michelle Noorali, and Adam Reese have "Storage Admin" access. There's also a chart uploader service account.

 

To add some context about cost and history...

We have seen numbers on what it would cost to host these through Google Cloud. The numbers, if I understood what I saw right, is the public cost rather than the internal cross charge cost. It's a larger number than I ever expected and a cost I'm very hesitant to ask the CNCF to cover as an unexpected expense. Especially during these times.

If there were a bunch of unused Google Cloud Credits laying around I would be happy to use them. But, that doesn't appear to be the case.

Just know, the cost to the CNCF would be substantial.

One of the problems is the hard coded URLs in the Helm client. This was all setup in early 2016 when Jack Greenfield and Ville Aikas were major Helm contributors on behalf of Google. The bucket setup (as with all the Helm Google Cloud services) were setup through Google corp accounts. Googlers controlled them and do to this day.

The lack of ownership by the project has been a pain point. But, older Helm v2 clients are stuck with it (can't do a redirect) and many of them are still in use. The version use across Helm versions is rather large.

What we want is to deprecate the charts repository (headed toward a distributed route) while still keeping things available for people. That includes old versions of charts.

I hope that context helps.

- Matt Farina

On Wed, Oct 14, 2020, at 2:10 PM, Josh Berkus wrote:
On 10/14/20 10:52 AM, Matt Farina wrote:

> There are two things worth noting...

>  1. Many of the charts people use have been slower to move out of the
>     stable and incubator repos than expected. We are still finding them
>     new homes. So, not everything is deprecated. There are many useful
>     things there.
>  2. There is more use of the stable and incubator repos than I ever
>     expected. I'm told the use is growing even as things move off it.
>     When I recently saw some numbers I was amazingly surprised. I had
>     expected it to be lower. Note, I don't have access to this data. The
>     buckets are Google owned/managed so only they have access to this. I
>     think Vic is the only one of the Helm maintainers with access to this.

> Many people will have some pain when the stable and incubator repos go away.

My question is: is the *Helm project* asking for the CNCF to fund
extended storage for these charts, or not? (assuming this is technically
possible).  I can see why the answer might be "not"; we have break the
links to the old locations sometime, and why not now?

The request to the TOC came from a concerned contributor, not from the
project leadership, as far as I can tell.

For my part, I would not want to see the TOC messing with a project's
official deprecation plans without a formal project request.

-- 
--
Josh Berkus
Kubernetes Community
Red Hat OSPO




Re: [URGENT] Impending deletion of Helm 2 releases and charts buckets

Matt Farina
 


From what I have been told, it's not technically possible to transfer the project.

That's not correct. Who told you that? As I pointed out on https://github.com/helm/community/issues/114:

Transferring the GCP Project to a CNCF GCP Organization and Account would make it easier for you to manage, and has to be done if it's going to continue to exist.

Sorry for the confusion. I've asked multiple people at Google about this and I was told it wasn't possible. Thank you for not only clearing up the confusion but pointing out the project-migration docs.

This still doesn't deal with the funding issue. We've not gone to the GB on funding (it would be well over the limit to just do so GB is needed) because the amount would be high enough that it impacts other CNCF activities like security audits and contractors working on things. We don't want to see that happen.

Now, if Helm could use some of the unused GCP credits for k8s the funding wouldn't be a problem. There are more than enough unused credits. Is transferring the bucket to the CNCF k8s account and setting it up to use some of those credits possible? We would shrink down what the buckets served and it would enable the k8s SIG Service catalog Helm repo to stay alive as it's managed through the same project.

You, Michelle Noorali, and Adam Reese have "Storage Admin" access. There's also a chart uploader service account.

This highlights part of the problem. If we look at the current Helm governance this isn't a list of the right people. Sadly, my access didn't even let me see who the other people were or that my access was "Storage Admin". IAM is a finicky thing.

The lack of ownership is more than a storage problem. For example, not having access to setup a custom domain with HTTPS. You need more than storage admin for that. This is what I meant by a pain point. Or, being able to manage who has access. Of the charts maintainers, I'm the only one on that list. Adam and Michelle aren't. What if I'm unavailable and someone else needs to step in? The reality is that "storage admin" isn't enough to really manage something well. That's what I meant.

- Matt Farina

On Wed, Oct 14, 2020, at 2:53 PM, Brian Grant wrote:
On Wed, Oct 14, 2020 at 11:50 AM Matt Farina <matt@...> wrote:

From what I have been told, it's not technically possible to transfer the project.

That's not correct. Who told you that? As I pointed out on https://github.com/helm/community/issues/114:

Transferring the GCP Project to a CNCF GCP Organization and Account would make it easier for you to manage, and has to be done if it's going to continue to exist.


You, Michelle Noorali, and Adam Reese have "Storage Admin" access. There's also a chart uploader service account.

 


To add some context about cost and history...

We have seen numbers on what it would cost to host these through Google Cloud. The numbers, if I understood what I saw right, is the public cost rather than the internal cross charge cost. It's a larger number than I ever expected and a cost I'm very hesitant to ask the CNCF to cover as an unexpected expense. Especially during these times.

If there were a bunch of unused Google Cloud Credits laying around I would be happy to use them. But, that doesn't appear to be the case.

Just know, the cost to the CNCF would be substantial.

One of the problems is the hard coded URLs in the Helm client. This was all setup in early 2016 when Jack Greenfield and Ville Aikas were major Helm contributors on behalf of Google. The bucket setup (as with all the Helm Google Cloud services) were setup through Google corp accounts. Googlers controlled them and do to this day.

The lack of ownership by the project has been a pain point. But, older Helm v2 clients are stuck with it (can't do a redirect) and many of them are still in use. The version use across Helm versions is rather large.

What we want is to deprecate the charts repository (headed toward a distributed route) while still keeping things available for people. That includes old versions of charts.

I hope that context helps.

- Matt Farina

On Wed, Oct 14, 2020, at 2:10 PM, Josh Berkus wrote:
On 10/14/20 10:52 AM, Matt Farina wrote:

> There are two things worth noting...

>  1. Many of the charts people use have been slower to move out of the
>     stable and incubator repos than expected. We are still finding them
>     new homes. So, not everything is deprecated. There are many useful
>     things there.
>  2. There is more use of the stable and incubator repos than I ever
>     expected. I'm told the use is growing even as things move off it.
>     When I recently saw some numbers I was amazingly surprised. I had
>     expected it to be lower. Note, I don't have access to this data. The
>     buckets are Google owned/managed so only they have access to this. I
>     think Vic is the only one of the Helm maintainers with access to this.

> Many people will have some pain when the stable and incubator repos go away.

My question is: is the *Helm project* asking for the CNCF to fund
extended storage for these charts, or not? (assuming this is technically
possible).  I can see why the answer might be "not"; we have break the
links to the old locations sometime, and why not now?

The request to the TOC came from a concerned contributor, not from the
project leadership, as far as I can tell.

For my part, I would not want to see the TOC messing with a project's
official deprecation plans without a formal project request.

-- 
--
Josh Berkus
Kubernetes Community
Red Hat OSPO







Re: [URGENT] Impending deletion of Helm 2 releases and charts buckets

Brian Grant
 

On Wed, Oct 14, 2020 at 11:50 AM Matt Farina <matt@...> wrote:
From what I have been told, it's not technically possible to transfer the project.

That's not correct. Who told you that? As I pointed out on https://github.com/helm/community/issues/114:

Transferring the GCP Project to a CNCF GCP Organization and Account would make it easier for you to manage, and has to be done if it's going to continue to exist.

You, Michelle Noorali, and Adam Reese have "Storage Admin" access. There's also a chart uploader service account.

 

To add some context about cost and history...

We have seen numbers on what it would cost to host these through Google Cloud. The numbers, if I understood what I saw right, is the public cost rather than the internal cross charge cost. It's a larger number than I ever expected and a cost I'm very hesitant to ask the CNCF to cover as an unexpected expense. Especially during these times.

If there were a bunch of unused Google Cloud Credits laying around I would be happy to use them. But, that doesn't appear to be the case.

Just know, the cost to the CNCF would be substantial.

One of the problems is the hard coded URLs in the Helm client. This was all setup in early 2016 when Jack Greenfield and Ville Aikas were major Helm contributors on behalf of Google. The bucket setup (as with all the Helm Google Cloud services) were setup through Google corp accounts. Googlers controlled them and do to this day.

The lack of ownership by the project has been a pain point. But, older Helm v2 clients are stuck with it (can't do a redirect) and many of them are still in use. The version use across Helm versions is rather large.

What we want is to deprecate the charts repository (headed toward a distributed route) while still keeping things available for people. That includes old versions of charts.

I hope that context helps.

- Matt Farina

On Wed, Oct 14, 2020, at 2:10 PM, Josh Berkus wrote:
On 10/14/20 10:52 AM, Matt Farina wrote:

> There are two things worth noting...

>  1. Many of the charts people use have been slower to move out of the
>     stable and incubator repos than expected. We are still finding them
>     new homes. So, not everything is deprecated. There are many useful
>     things there.
>  2. There is more use of the stable and incubator repos than I ever
>     expected. I'm told the use is growing even as things move off it.
>     When I recently saw some numbers I was amazingly surprised. I had
>     expected it to be lower. Note, I don't have access to this data. The
>     buckets are Google owned/managed so only they have access to this. I
>     think Vic is the only one of the Helm maintainers with access to this.

> Many people will have some pain when the stable and incubator repos go away.

My question is: is the *Helm project* asking for the CNCF to fund
extended storage for these charts, or not? (assuming this is technically
possible).  I can see why the answer might be "not"; we have break the
links to the old locations sometime, and why not now?

The request to the TOC came from a concerned contributor, not from the
project leadership, as far as I can tell.

For my part, I would not want to see the TOC messing with a project's
official deprecation plans without a formal project request.

-- 
--
Josh Berkus
Kubernetes Community
Red Hat OSPO




Re: [URGENT] Impending deletion of Helm 2 releases and charts buckets

Matt Farina
 

From what I have been told, it's not technically possible to transfer the project.

To add some context about cost and history...

We have seen numbers on what it would cost to host these through Google Cloud. The numbers, if I understood what I saw right, is the public cost rather than the internal cross charge cost. It's a larger number than I ever expected and a cost I'm very hesitant to ask the CNCF to cover as an unexpected expense. Especially during these times.

If there were a bunch of unused Google Cloud Credits laying around I would be happy to use them. But, that doesn't appear to be the case.

Just know, the cost to the CNCF would be substantial.

One of the problems is the hard coded URLs in the Helm client. This was all setup in early 2016 when Jack Greenfield and Ville Aikas were major Helm contributors on behalf of Google. The bucket setup (as with all the Helm Google Cloud services) were setup through Google corp accounts. Googlers controlled them and do to this day.

The lack of ownership by the project has been a pain point. But, older Helm v2 clients are stuck with it (can't do a redirect) and many of them are still in use. The version use across Helm versions is rather large.

What we want is to deprecate the charts repository (headed toward a distributed route) while still keeping things available for people. That includes old versions of charts.

I hope that context helps.

- Matt Farina

On Wed, Oct 14, 2020, at 2:10 PM, Josh Berkus wrote:
On 10/14/20 10:52 AM, Matt Farina wrote:

> There are two things worth noting...

>  1. Many of the charts people use have been slower to move out of the
>     stable and incubator repos than expected. We are still finding them
>     new homes. So, not everything is deprecated. There are many useful
>     things there.
>  2. There is more use of the stable and incubator repos than I ever
>     expected. I'm told the use is growing even as things move off it.
>     When I recently saw some numbers I was amazingly surprised. I had
>     expected it to be lower. Note, I don't have access to this data. The
>     buckets are Google owned/managed so only they have access to this. I
>     think Vic is the only one of the Helm maintainers with access to this.

> Many people will have some pain when the stable and incubator repos go away.

My question is: is the *Helm project* asking for the CNCF to fund
extended storage for these charts, or not? (assuming this is technically
possible).  I can see why the answer might be "not"; we have break the
links to the old locations sometime, and why not now?

The request to the TOC came from a concerned contributor, not from the
project leadership, as far as I can tell.

For my part, I would not want to see the TOC messing with a project's
official deprecation plans without a formal project request.

-- 
--
Josh Berkus
Kubernetes Community
Red Hat OSPO




Re: [EXTERNAL] Re: [cncf-toc] [URGENT] Impending deletion of Helm 2 releases and charts buckets

Matt Butcher <Matt.Butcher@...>
 

All we need from Google is for them to redirect the chart repository domain name to a new location. That is the only thing that is currently causing the high-probability risk of the chart repositories experiencing an outage. Who at Google can do that? The original setup was done by Jack Greenfield. I don't believe he still works on Google Cloud. I do not even know who currently "owns" the cloud account.

Matt


From: Brendan Burns <bburns@...>
Sent: Wednesday, October 14, 2020 8:14 AM
To: Liz Rice <liz@...>; briangrant@... <briangrant@...>; Matt Butcher <Matt.Butcher@...>
Cc: Chris Aniszczyk <caniszczyk@...>; Matt Farina <matt@...>; agupta <agupta@...>; Michelle Noorali <Michelle.Noorali@...>; Priyanka Sharma <psharma@...>; cncf-toc@... <cncf-toc@...>; Saad Ali <saadali@...>
Subject: Re: [EXTERNAL] Re: [cncf-toc] [URGENT] Impending deletion of Helm 2 releases and charts buckets
 
+MattButcher

--brendan


From: cncf-toc@... <cncf-toc@...> on behalf of Brian Grant via lists.cncf.io <briangrant=google.com@...>
Sent: Wednesday, October 14, 2020 7:05:41 AM
To: Liz Rice <liz@...>
Cc: Chris Aniszczyk <caniszczyk@...>; Matt Farina <matt@...>; agupta <agupta@...>; Michelle Noorali <Michelle.Noorali@...>; Priyanka Sharma <psharma@...>; cncf-toc@... <cncf-toc@...>; Saad Ali <saadali@...>
Subject: [EXTERNAL] Re: [cncf-toc] [URGENT] Impending deletion of Helm 2 releases and charts buckets
 
On Wed, Oct 14, 2020 at 7:02 AM Liz Rice <liz@...> wrote:
Thanks Chris. Just noting that we do have the Special Issues process where $$$ are involved to take technical budget issues to the GB, though it sounds unlikely that this would be an affordable item for the community to sustain. (I am wondering if there is a stop-gap measure to fund the storage for a short period of time while the deprecation message gets through to users, to stop it falling off a cliff - but there is only any point in doing that if we can effectively encourage migration off of these resources.)

This has been an issue since at least April 2019. The deprecation message is not getting through.
 

On Wed, Oct 14, 2020 at 2:20 PM Chris Aniszczyk <caniszczyk@...> wrote:
It would be great to have input from the current Helm maintainers on their issue and public deprecation plan here. +Matt Farina 

Also, this is a big budget item and will likely involve the GB, where the TOC should mostly focus on the technical deprecation plan and how to do this for the overall ecosystem with minimal impact.  As I understand it, the storage cost of the charts is relatively cheap but the bandwidth costs are considerable and would make up a huge chunk of the budget and honestly something usually sponsored by large cloud providers or a CDN, similar to how we handle the Kubernetes project or even other projects like say jquery.

 A simple solution would be to reuse or extend the existing credits that exist for Kubernetes [1] and firm up an ownership plan with the Helm maintainers on when deprecation can happen with a firm date, but this requires collaboration with the Helm maintainers [2] and the wider community [3]. We can also try to offer some time at the upcoming KubeCon to further make the community aware of the upcoming deprecation, but these things are always tricky for when it comes to widely used software.


On Wed, Oct 14, 2020 at 6:33 AM Brian Grant <briangrant@...> wrote:
On Wed, Oct 14, 2020 at 12:51 AM Liz Rice <liz@...> wrote:
It does seem like a hard cut-off is going to hurt people without some further action. I can see two ways the CNCF might be able to help:

- Getting the word out. I suspect that a lot of Helm chart creation is done by folks outside of the "Helm community" and maybe the message is not reaching the wider cloud world. This isn't a criticism of the Helm community or the way it communicates, it's an indicator of how widely used the project is. I think CNCF could help push a message to this wider audience

The affected charts are mainly the ones in the official community repo:

There is a notice in the README. 

PRs to the charts are still being merged, though at a lower rate than earlier in the year:


- I'm not clear from the GH thread whether there is already a CNCF owned location that the old locations can be redirected to, but I'd be supportive of CNCF funds being used for that purpose. Is it more-or-less as simple as redirecting URLs from google-funded resources to community-funded resources? 

There is not a CNCF-owned location that I'm aware of.

Regarding redirects:
"The URL to the stable repository has been embedded in Helm since v2.0.0-alpha.2. It's directly to a Google Cloud bucket"

No redirector was put in place.

The project containing the buckets could be moved to a CNCF account.

Does Artifact Hub have a role to play here? 

Finally, I can't be the first person to think this thought, but is there some possible way to contact chart owners and tell them about the imminent demise of the current buckets? 

Liz 


On Wed, 14 Oct 2020 at 07:45, Arvind Gupta <agupta@...> wrote:
I would recommend setting up a URL forwarding from old repos to new repos, so that charts are maintained at new locations but old URL references continue to work. 

Thanks,
-Arvind


On Tue, Oct 13, 2020 at 11:24 PM Saad Ali via lists.cncf.io <saadali=google.com@...> wrote:
TOC Folks,

I want to bring https://github.com/helm/community/issues/114 to your attention.
This has the potential for causing a lot of pain to users.

The buckets are scheduled to be deleted on Nov 13, 2020, per the original deprecation plan. As that date is fast approaching, it is becoming clear that doing so could cause major issues. Per Brian's comment on the bug "The usage of the charts storage buckets hasn't decreased at all. It's higher than when the deprecation was announced last October. There have been about 3.5 accesses per second on average over the last 12 hours."

The Helm project has been trying very hard to get the word out, migrating what they can, and pointing people to chartcenter.io (which has copies of both incubator and stable as an alternative).
But the hard part to fight is the countless posts/examples on stackoverflow/other sites, and the many vendors who apparently haven't migrated yet.

Given that and the bucket usage metrics, the fear is that the number of people still using the old repos may be under-estimated, and shutting down the buckets could cause many unanticipated outages. The long lead time was supposed to be about getting users off, but that result does not appear to be achieved yet.

As TOC, should we consider alternative options such as transitioning ownership of these resources to CNCF to ensure a longer runway for deprecation?

Regards,

Saad Ali



--
Chris Aniszczyk (@cra)


Re: [URGENT] Impending deletion of Helm 2 releases and charts buckets

Josh Berkus
 

On 10/14/20 10:52 AM, Matt Farina wrote:

There are two things worth noting...

1. Many of the charts people use have been slower to move out of the
stable and incubator repos than expected. We are still finding them
new homes. So, not everything is deprecated. There are many useful
things there.
2. There is more use of the stable and incubator repos than I ever
expected. I'm told the use is growing even as things move off it.
When I recently saw some numbers I was amazingly surprised. I had
expected it to be lower. Note, I don't have access to this data. The
buckets are Google owned/managed so only they have access to this. I
think Vic is the only one of the Helm maintainers with access to this.

Many people will have some pain when the stable and incubator repos go away.
My question is: is the *Helm project* asking for the CNCF to fund
extended storage for these charts, or not? (assuming this is technically
possible). I can see why the answer might be "not"; we have break the
links to the old locations sometime, and why not now?

The request to the TOC came from a concerned contributor, not from the
project leadership, as far as I can tell.

For my part, I would not want to see the TOC messing with a project's
official deprecation plans without a formal project request.

--
--
Josh Berkus
Kubernetes Community
Red Hat OSPO


Re: [URGENT] Impending deletion of Helm 2 releases and charts buckets

Matt Farina
 

Josh,

You ask a good question.

There are two things worth noting...
  1. Many of the charts people use have been slower to move out of the stable and incubator repos than expected. We are still finding them new homes. So, not everything is deprecated. There are many useful things there.
  2. There is more use of the stable and incubator repos than I ever expected. I'm told the use is growing even as things move off it. When I recently saw some numbers I was amazingly surprised. I had expected it to be lower. Note, I don't have access to this data. The buckets are Google owned/managed so only they have access to this. I think Vic is the only one of the Helm maintainers with access to this.
Many people will have some pain when the stable and incubator repos go away.

We do have some announcements (I'm hoping next week) and a path forward we're working out that will help make this easier.

- Matt Farina


On Wed, Oct 14, 2020, at 1:18 PM, Josh Berkus wrote:
All,

I'm really unclear on why CNCF would devote resources to maintaining a
set of outdated Helm charts if the Helm project itself has decided not
to do so.  Wouldn't it be more sensible for us to use the CNCF marketing
resources to make sure that people know about the impending scheduled
removal?

-- 
--
Josh Berkus
Kubernetes Community
Red Hat OSPO




Re: [URGENT] Impending deletion of Helm 2 releases and charts buckets

Josh Berkus
 

All,

I'm really unclear on why CNCF would devote resources to maintaining a
set of outdated Helm charts if the Helm project itself has decided not
to do so. Wouldn't it be more sensible for us to use the CNCF marketing
resources to make sure that people know about the impending scheduled
removal?

--
--
Josh Berkus
Kubernetes Community
Red Hat OSPO


Re: [EXTERNAL] Re: [cncf-toc] [URGENT] Impending deletion of Helm 2 releases and charts buckets

Brendan Burns
 

+MattButcher

--brendan


From: cncf-toc@... <cncf-toc@...> on behalf of Brian Grant via lists.cncf.io <briangrant=google.com@...>
Sent: Wednesday, October 14, 2020 7:05:41 AM
To: Liz Rice <liz@...>
Cc: Chris Aniszczyk <caniszczyk@...>; Matt Farina <matt@...>; agupta <agupta@...>; Michelle Noorali <Michelle.Noorali@...>; Priyanka Sharma <psharma@...>; cncf-toc@... <cncf-toc@...>; Saad Ali <saadali@...>
Subject: [EXTERNAL] Re: [cncf-toc] [URGENT] Impending deletion of Helm 2 releases and charts buckets
 
On Wed, Oct 14, 2020 at 7:02 AM Liz Rice <liz@...> wrote:
Thanks Chris. Just noting that we do have the Special Issues process where $$$ are involved to take technical budget issues to the GB, though it sounds unlikely that this would be an affordable item for the community to sustain. (I am wondering if there is a stop-gap measure to fund the storage for a short period of time while the deprecation message gets through to users, to stop it falling off a cliff - but there is only any point in doing that if we can effectively encourage migration off of these resources.)

This has been an issue since at least April 2019. The deprecation message is not getting through.
 

On Wed, Oct 14, 2020 at 2:20 PM Chris Aniszczyk <caniszczyk@...> wrote:
It would be great to have input from the current Helm maintainers on their issue and public deprecation plan here. +Matt Farina 

Also, this is a big budget item and will likely involve the GB, where the TOC should mostly focus on the technical deprecation plan and how to do this for the overall ecosystem with minimal impact.  As I understand it, the storage cost of the charts is relatively cheap but the bandwidth costs are considerable and would make up a huge chunk of the budget and honestly something usually sponsored by large cloud providers or a CDN, similar to how we handle the Kubernetes project or even other projects like say jquery.

 A simple solution would be to reuse or extend the existing credits that exist for Kubernetes [1] and firm up an ownership plan with the Helm maintainers on when deprecation can happen with a firm date, but this requires collaboration with the Helm maintainers [2] and the wider community [3]. We can also try to offer some time at the upcoming KubeCon to further make the community aware of the upcoming deprecation, but these things are always tricky for when it comes to widely used software.


On Wed, Oct 14, 2020 at 6:33 AM Brian Grant <briangrant@...> wrote:
On Wed, Oct 14, 2020 at 12:51 AM Liz Rice <liz@...> wrote:
It does seem like a hard cut-off is going to hurt people without some further action. I can see two ways the CNCF might be able to help:

- Getting the word out. I suspect that a lot of Helm chart creation is done by folks outside of the "Helm community" and maybe the message is not reaching the wider cloud world. This isn't a criticism of the Helm community or the way it communicates, it's an indicator of how widely used the project is. I think CNCF could help push a message to this wider audience

The affected charts are mainly the ones in the official community repo:

There is a notice in the README. 

PRs to the charts are still being merged, though at a lower rate than earlier in the year:


- I'm not clear from the GH thread whether there is already a CNCF owned location that the old locations can be redirected to, but I'd be supportive of CNCF funds being used for that purpose. Is it more-or-less as simple as redirecting URLs from google-funded resources to community-funded resources? 

There is not a CNCF-owned location that I'm aware of.

Regarding redirects:
"The URL to the stable repository has been embedded in Helm since v2.0.0-alpha.2. It's directly to a Google Cloud bucket"

No redirector was put in place.

The project containing the buckets could be moved to a CNCF account.

Does Artifact Hub have a role to play here? 

Finally, I can't be the first person to think this thought, but is there some possible way to contact chart owners and tell them about the imminent demise of the current buckets? 

Liz 


On Wed, 14 Oct 2020 at 07:45, Arvind Gupta <agupta@...> wrote:
I would recommend setting up a URL forwarding from old repos to new repos, so that charts are maintained at new locations but old URL references continue to work. 

Thanks,
-Arvind


On Tue, Oct 13, 2020 at 11:24 PM Saad Ali via lists.cncf.io <saadali=google.com@...> wrote:
TOC Folks,

I want to bring https://github.com/helm/community/issues/114 to your attention.
This has the potential for causing a lot of pain to users.

The buckets are scheduled to be deleted on Nov 13, 2020, per the original deprecation plan. As that date is fast approaching, it is becoming clear that doing so could cause major issues. Per Brian's comment on the bug "The usage of the charts storage buckets hasn't decreased at all. It's higher than when the deprecation was announced last October. There have been about 3.5 accesses per second on average over the last 12 hours."

The Helm project has been trying very hard to get the word out, migrating what they can, and pointing people to chartcenter.io (which has copies of both incubator and stable as an alternative).
But the hard part to fight is the countless posts/examples on stackoverflow/other sites, and the many vendors who apparently haven't migrated yet.

Given that and the bucket usage metrics, the fear is that the number of people still using the old repos may be under-estimated, and shutting down the buckets could cause many unanticipated outages. The long lead time was supposed to be about getting users off, but that result does not appear to be achieved yet.

As TOC, should we consider alternative options such as transitioning ownership of these resources to CNCF to ensure a longer runway for deprecation?

Regards,

Saad Ali



--
Chris Aniszczyk (@cra)


Re: [URGENT] Impending deletion of Helm 2 releases and charts buckets

Brian Grant
 

On Wed, Oct 14, 2020 at 7:02 AM Liz Rice <liz@...> wrote:
Thanks Chris. Just noting that we do have the Special Issues process where $$$ are involved to take technical budget issues to the GB, though it sounds unlikely that this would be an affordable item for the community to sustain. (I am wondering if there is a stop-gap measure to fund the storage for a short period of time while the deprecation message gets through to users, to stop it falling off a cliff - but there is only any point in doing that if we can effectively encourage migration off of these resources.)

This has been an issue since at least April 2019. The deprecation message is not getting through.
 

On Wed, Oct 14, 2020 at 2:20 PM Chris Aniszczyk <caniszczyk@...> wrote:
It would be great to have input from the current Helm maintainers on their issue and public deprecation plan here. +Matt Farina 

Also, this is a big budget item and will likely involve the GB, where the TOC should mostly focus on the technical deprecation plan and how to do this for the overall ecosystem with minimal impact.  As I understand it, the storage cost of the charts is relatively cheap but the bandwidth costs are considerable and would make up a huge chunk of the budget and honestly something usually sponsored by large cloud providers or a CDN, similar to how we handle the Kubernetes project or even other projects like say jquery.

 A simple solution would be to reuse or extend the existing credits that exist for Kubernetes [1] and firm up an ownership plan with the Helm maintainers on when deprecation can happen with a firm date, but this requires collaboration with the Helm maintainers [2] and the wider community [3]. We can also try to offer some time at the upcoming KubeCon to further make the community aware of the upcoming deprecation, but these things are always tricky for when it comes to widely used software.


On Wed, Oct 14, 2020 at 6:33 AM Brian Grant <briangrant@...> wrote:
On Wed, Oct 14, 2020 at 12:51 AM Liz Rice <liz@...> wrote:
It does seem like a hard cut-off is going to hurt people without some further action. I can see two ways the CNCF might be able to help:

- Getting the word out. I suspect that a lot of Helm chart creation is done by folks outside of the "Helm community" and maybe the message is not reaching the wider cloud world. This isn't a criticism of the Helm community or the way it communicates, it's an indicator of how widely used the project is. I think CNCF could help push a message to this wider audience

The affected charts are mainly the ones in the official community repo:

There is a notice in the README. 

PRs to the charts are still being merged, though at a lower rate than earlier in the year:


- I'm not clear from the GH thread whether there is already a CNCF owned location that the old locations can be redirected to, but I'd be supportive of CNCF funds being used for that purpose. Is it more-or-less as simple as redirecting URLs from google-funded resources to community-funded resources? 

There is not a CNCF-owned location that I'm aware of.

Regarding redirects:
"The URL to the stable repository has been embedded in Helm since v2.0.0-alpha.2. It's directly to a Google Cloud bucket"

No redirector was put in place.

The project containing the buckets could be moved to a CNCF account.

Does Artifact Hub have a role to play here? 

Finally, I can't be the first person to think this thought, but is there some possible way to contact chart owners and tell them about the imminent demise of the current buckets? 

Liz 


On Wed, 14 Oct 2020 at 07:45, Arvind Gupta <agupta@...> wrote:
I would recommend setting up a URL forwarding from old repos to new repos, so that charts are maintained at new locations but old URL references continue to work. 

Thanks,
-Arvind


On Tue, Oct 13, 2020 at 11:24 PM Saad Ali via lists.cncf.io <saadali=google.com@...> wrote:
TOC Folks,

I want to bring https://github.com/helm/community/issues/114 to your attention.
This has the potential for causing a lot of pain to users.

The buckets are scheduled to be deleted on Nov 13, 2020, per the original deprecation plan. As that date is fast approaching, it is becoming clear that doing so could cause major issues. Per Brian's comment on the bug "The usage of the charts storage buckets hasn't decreased at all. It's higher than when the deprecation was announced last October. There have been about 3.5 accesses per second on average over the last 12 hours."

The Helm project has been trying very hard to get the word out, migrating what they can, and pointing people to chartcenter.io (which has copies of both incubator and stable as an alternative).
But the hard part to fight is the countless posts/examples on stackoverflow/other sites, and the many vendors who apparently haven't migrated yet.

Given that and the bucket usage metrics, the fear is that the number of people still using the old repos may be under-estimated, and shutting down the buckets could cause many unanticipated outages. The long lead time was supposed to be about getting users off, but that result does not appear to be achieved yet.

As TOC, should we consider alternative options such as transitioning ownership of these resources to CNCF to ensure a longer runway for deprecation?

Regards,

Saad Ali



--
Chris Aniszczyk (@cra)