Replying top-level as my thoughts jump across the thread.


I didn't run the numbers, yet I believe that the pace of submissions
has picked up. That alone can increase backlog.

We tried SIGs (now TAGs) doing due diligence for projects. The level
of scrutiny, and the closeness to the guidance material available, was
different across TAGs. In effect, this meant inconsistent processes
which is arguably unfair. And in cases of disagreements, TOC is pulled
in automatically anyway.
A clear delegation from TOC might be possible, yet project advancement
is one of the main tasks of TOC and arguably what votees expect TOC to
do. In any case, it does change any of the underlying desires.

What TAGs could provide is an initial proving ground, though: Projects
could give a presentation and go through questions and feedback in a
more limited scope, allowing them to polish their submittal.


While I know that the current sandbox process is designed to be very
low barrier, I am still not convinced that this is an obviously
desirable design goal. It is true that a neutral playing field is good
and helps some projects grow. It is also true that "CNCF project"
holds immense marketing value and many efforts are ephemeral, in
particular if largely driven by perf & marketing.
Back when sandbox criteria were relaxed, I was of the opinion that
they should remain more stringent. I have come to wonder if four
levels wouldn't be more appropriate: An initial runway on which
projects can be put; but also pruned more aggressively if they do not
show growth/adoption/the usual. E.g. once submitted they have three?
six? twelve? months to show certain progress or are removed outright.
Medium term, this might also allow for a smaller jump towards
Incubating, which is currently significant.


Orthogonally, I believe we can manage expectations better. One
possible approach would be to create dashboards and reports of the
underlying data to help manage expectations and keep ourselves honest.
What are the average and median times a project takes from stage X to
stage Y? How has this changed over time?
Another would be to rework the process & documentation; e.g.
Incubation had distinct requirement docs which TAGs copied together
and deduplicated back during the DD trials.



Having seen things from both sides now, and since CNCF started, I can
understand both the frustrations about some timelines better and also
understand how a few dedicated people are trying to do their best with
the time they have. On all sides.


Best,
Richard

+1 NB

Chris Short

He/Him/His

Sr. Developer Advocate, AWS Kubernetes (GitOps)

TZ=America/Detroit

On Apr 26, 2022, at 22:54, Rahul Jadhav <r@...> wrote:

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.

+1 NB

On Tue, Apr 26, 2022 at 4:56 PM Davanum Srinivas <davanum@...> wrote:

Hi Folks,

Kyverno has applied to move from sandbox to incubation. As the TOC sponsor, thanks to everyone for their work.

PR: https://github.com/cncf/toc/pull/784
DD: https://docs.google.com/document/d/18dWgOd2MUQz3RXI1R9vKntL3ULyZhOD1HEtijGOeaWg/edit?usp=sharing

Everyone is welcome to comment in the document, on the PR, or in reply to this thread, before we move to a TOC vote. This period of public comment will last a minimum of two weeks.

Thanks,

Dims

-- 

Davanum Srinivas :: https://twitter.com/dims

Katie,

typically whoever makes it to the event in-person :)

On Tue, May 3, 2022 at 12:05 AM Katie Gamanji <gamanjie@...> wrote:

Thank you Amye for the update!

Do we know who will represent the TOC in the panel?

On Mon, May 2, 2022 at 10:08 PM Amye Scavarda Perrin <ascavarda@...> wrote:

A quick note that the TOC meetings for May 3rd and May 17th are cancelled. We have a conflict with the TOC for the 3rd, and we'll have an open TOC panel at KubeCon replacing our standard meeting for May 17th. 

Wednesday, May 18 • 15:25 - 16:00 Central European Summer Time

https://sched.co/ytmW

--

Amye Scavarda Perrin | Director of Developer Programs, CNCF | amye@...

--

Davanum Srinivas :: https://twitter.com/dims

Thank you Amye for the update!

Do we know who will represent the TOC in the panel?

On Mon, May 2, 2022 at 10:08 PM Amye Scavarda Perrin <ascavarda@...> wrote:

A quick note that the TOC meetings for May 3rd and May 17th are cancelled. We have a conflict with the TOC for the 3rd, and we'll have an open TOC panel at KubeCon replacing our standard meeting for May 17th. 

Wednesday, May 18 • 15:25 - 16:00 Central European Summer Time

https://sched.co/ytmW

--

Amye Scavarda Perrin | Director of Developer Programs, CNCF | amye@...

Hi Maulik,

 

I’m not on the TOC, so I’m not sure if there were specific concerns about Matos raised during the meeting, but I do have a few suggestions for you based on our work within TAG Contributor Strategy.

 

I recommend having a look at some of the TAG Contributor Strategy resources about Contributor Growth here: https://contribute.cncf.io/maintainers/community/contributor-growth-framework/

 

I also recommend completing some of the TODO items in your Contribution Guidelines documentation: https://github.com/cloudmatos/Matos/blob/main/docs/CONTRIBUTION_GUIDELINES.md

 

Right now, I think most people would find it challenging to contribute without instructions for building Matos locally and running tests. The easier you can make it for contributors to get started, the easier it will be to recruit contributors.

 

Cheers,

Dawn

 

From: cncf-toc@... <cncf-toc@...> on behalf of Maulik Shyani via lists.cncf.io <maulik=cloudmatos.com@...>
Date: Monday, May 2, 2022 at 9:02 PM
To: Amye Scavarda Perrin <ascavarda@...>, CNCF TOC <cncf-toc@...>
Subject: Re: [cncf-toc] Results from Sandbox Inclusion Meeting, April 26

⚠ External Email

Hello Amye and TOC,

 

Thanks for the details below on our Sandbox project entry of Matos. 

 

Decision: Matos: Reapply in January '23 showing more robust community.

 

We submitted our entry in December'21 and got selected at the end of April'22 to be reviewed but we were informed to reapply in Jan'23. 

 

I completely understand that there are so many applications that want to be part of the Sandbox project and that's why it's taking time to be reviewed but my understanding is that the Sandbox project will provide better visibility to the community to participate in the open source project. 

 

How do we build the community without being part of the Sandbox or having better visibility?Would you please share any recommendations on how to build a robust community?

 

I appreciate your help here!

Thanks!  

 

On Tue, Apr 26, 2022 at 11:25 AM Amye Scavarda Perrin <ascavarda@...> wrote:

The TOC met today to review the sandbox applications available at sandbox.cncf.io.

OpenFunction - passes with a majority vote of the TOC
Teller - passes with a majority vote of the TOC
sealer - passes with a majority vote of the TOC

Our next Sandbox review meeting is June 14.

Not included at the sandbox level:

Ketch: Reapply with a more robust community presence.
container-structure-test: Reapply with a more robust community presence.
Clusternet: TOC would like to see more people who are active in the project actively doing PRS and reviews and issues.
Tarian: Consider becoming a subproject of Falco
Kubescape: would like to see more community growth, TAG + SIG Security may be useful here, reapply in 6 months
Lagoon: Reapply in 6 months to a year.
Matos: Reapply in January '23 showing more robust community.
KTLS: Suggest meeting with Kubernetes SIG-Release to work together.
Cluster API Provider for CloudStack(CAPC): Suggest meeting with SIG Cluster Lifecycle.

 

--

Amye Scavarda Perrin | Director of Developer Programs, CNCF | amye@...

 

--

Thanks and Regards,

Maulik Shyani

 

 

 

 

---

⚠ External Email: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender.

A quick note that the TOC meetings for May 3rd and May 17th are cancelled. We have a conflict with the TOC for the 3rd, and we'll have an open TOC panel at KubeCon replacing our standard meeting for May 17th. 

Wednesday, May 18 • 15:25 - 16:00 Central European Summer Time

https://sched.co/ytmW

--

Amye Scavarda Perrin | Director of Developer Programs, CNCF | amye@...

The TOC met today to review the sandbox applications available at sandbox.cncf.io.

OpenFunction - passes with a majority vote of the TOC
Teller - passes with a majority vote of the TOC
sealer - passes with a majority vote of the TOC

Our next Sandbox review meeting is June 14.

Not included at the sandbox level:

Ketch: Reapply with a more robust community presence.
container-structure-test: Reapply with a more robust community presence.
Clusternet: TOC would like to see more people who are active in the project actively doing PRS and reviews and issues.
Tarian: Consider becoming a subproject of Falco
Kubescape: would like to see more community growth, TAG + SIG Security may be useful here, reapply in 6 months
Lagoon: Reapply in 6 months to a year.
Matos: Reapply in January '23 showing more robust community.
KTLS: Suggest meeting with Kubernetes SIG-Release to work together.
Cluster API Provider for CloudStack(CAPC): Suggest meeting with SIG Cluster Lifecycle.

--

Amye Scavarda Perrin | Director of Developer Programs, CNCF | amye@...

I should add, that’s not intended as a criticism -  the number of very early stage applications from individuals and single vendors has increased, which over time opened up the question for the TOC of whether it’s really right to commit CNCF resources for these projects. 

Those discussions naturally move us away from the original intention that the process should involve very little assessment or subjective judgement (e.g. the intention was to avoid a complicated definition of what is “mature enough” for sandbox) 

On Mon, 2 May 2022 at 18:48, Liz Rice <liz@...> wrote:

A little history: the current process was supposed to be super-lightweight, to reflect the very, very low bar for Sandbox projects - essentially, is it cloud native. I don’t remember the exact number but I’m pretty sure we got through a lot more than 12 applications in the first meeting.

Maybe it’s worth the TOC revisiting what that low bar really should be so that it’s easier and quicker to assess?  Here’s a suggestion that would make it super lightweight but I think still be in line with the CNCF mission. 

One of our reasons to exist is to enable multiple organisations to have a neutral place to collaborate, even if the project is little more than at the paper napkin stage. Based on this, we could define the bar for Sandbox as: a project needs to have support from minimum two CNCF member organisations who consider themselves stakeholders in the project. That could mean they’re involved in building it, or interested in using it. The onus is on the project to find those stakeholders before applying. The TOC’s approval would simply be a check that they agree that it’s a cloud native project and that they don’t have any other objection to it being included 

On Sun, 1 May 2022 at 19:29, alexis richardson <alexis@...> wrote:

+1 

On Sun, 1 May 2022, 19:27 Matt Farina, <matt@...> wrote:

I, for one, would love to see the sandbox process be faster and improve.

With regard to moving more work to the TAGs, two things come to mind.

First, when TAGs did more in the past they were inconsistent across each other and added their own criteria. This was a problem I don't want to see again. For example, I remember when one project lead was proposing a project for sandbox. He was frustrated because his project was criticized for not meeting a graduation criteria and for a criteria that was of the TAGs own making. I don't think we want this to happen again.

Second, sandbox projects don't get or need an in depth technical analysis. That shows up for incubation. I'm wondering, what would a TAG do here that wouldn't be repeated by the TOC when they go to look at it?

Having been through sandbox reviews twice now and having given advice to some projects that wanted to go for sandbox I've learned a few areas that could use some improvement...

1. I've answered a lot of questions about things not in the docs. Things that provide context to the CNCF, what sandbox is, what I think the TOC is looking for, and how to communicate well to the TOC. I think this could be better documented.
   
2. TAGs have a unique intersection where they have experts in an area and they work with the projects. I (and the rest of the TOC) don't scale on advising projects for sandbox. The TAGs may be able to do that. While I wouldn't require it, it could be useful for those who want to submit a sandbox project to present to the appropriate TAG and get guidance from them. For those who need it, getting some mentoring from a TAG could be useful.
   

These are just my initial thoughts. Happy to hear agreement, disagreement, or things built upon this.

Cheers,

Matt

On Sun, May 1, 2022, at 11:09 AM, Matt Young wrote:

We have been thinking about this in TAG Observability as well, and have work in flight that’s related:

* Form Program: Annual Sandbox Review [1]

    * Create summary slides [2]

[1] https://github.com/cncf/tag-observability/issues/80

[2] 

https://github.com/cncf/tag-observability/issues/83

Will have details Tuesday as part of TAG update.

Matt

On Sun, May 1, 2022 at 9:41 AM Josh Gavant <joshgavant@...> wrote:

[Edited Message Follows]

+1 to pushing more to TAGs. Perhaps each proposed project can be assigned to a TAG and a member of the TAG can lead a technical review and guide the project's leads on criteria for acceptance. That could help TOC reviews go more smoothly, make them more likely to succeed, and ensure projects and contributors don't get lost or feel unsupported along the journey.

It could also give first-time CNCF/TAG contributors an idea of where to start - they could pick an open project for the TAG and review and present it to the group.

As a start in this direction a couple months ago I created a label in TAG App Delivery to track project review requests from TOC: https://github.com/cncf/tag-app-delivery/issues?q=label%3Atoc-review

A little history: the current process was supposed to be super-lightweight, to reflect the very, very low bar for Sandbox projects - essentially, is it cloud native. I don’t remember the exact number but I’m pretty sure we got through a lot more than 12 applications in the first meeting.

Maybe it’s worth the TOC revisiting what that low bar really should be so that it’s easier and quicker to assess?  Here’s a suggestion that would make it super lightweight but I think still be in line with the CNCF mission. 

One of our reasons to exist is to enable multiple organisations to have a neutral place to collaborate, even if the project is little more than at the paper napkin stage. Based on this, we could define the bar for Sandbox as: a project needs to have support from minimum two CNCF member organisations who consider themselves stakeholders in the project. That could mean they’re involved in building it, or interested in using it. The onus is on the project to find those stakeholders before applying. The TOC’s approval would simply be a check that they agree that it’s a cloud native project and that they don’t have any other objection to it being included 

On Sun, 1 May 2022 at 19:29, alexis richardson <alexis@...> wrote:

+1 

On Sun, 1 May 2022, 19:27 Matt Farina, <matt@...> wrote:

I, for one, would love to see the sandbox process be faster and improve.

With regard to moving more work to the TAGs, two things come to mind.

First, when TAGs did more in the past they were inconsistent across each other and added their own criteria. This was a problem I don't want to see again. For example, I remember when one project lead was proposing a project for sandbox. He was frustrated because his project was criticized for not meeting a graduation criteria and for a criteria that was of the TAGs own making. I don't think we want this to happen again.

Second, sandbox projects don't get or need an in depth technical analysis. That shows up for incubation. I'm wondering, what would a TAG do here that wouldn't be repeated by the TOC when they go to look at it?

Having been through sandbox reviews twice now and having given advice to some projects that wanted to go for sandbox I've learned a few areas that could use some improvement...

1. I've answered a lot of questions about things not in the docs. Things that provide context to the CNCF, what sandbox is, what I think the TOC is looking for, and how to communicate well to the TOC. I think this could be better documented.
   
2. TAGs have a unique intersection where they have experts in an area and they work with the projects. I (and the rest of the TOC) don't scale on advising projects for sandbox. The TAGs may be able to do that. While I wouldn't require it, it could be useful for those who want to submit a sandbox project to present to the appropriate TAG and get guidance from them. For those who need it, getting some mentoring from a TAG could be useful.
   

These are just my initial thoughts. Happy to hear agreement, disagreement, or things built upon this.

Cheers,

Matt

On Sun, May 1, 2022, at 11:09 AM, Matt Young wrote:

We have been thinking about this in TAG Observability as well, and have work in flight that’s related:

* Form Program: Annual Sandbox Review [1]

    * Create summary slides [2]

[1] https://github.com/cncf/tag-observability/issues/80

[2] 

https://github.com/cncf/tag-observability/issues/83

Will have details Tuesday as part of TAG update.

Matt

On Sun, May 1, 2022 at 9:41 AM Josh Gavant <joshgavant@...> wrote:

[Edited Message Follows]

+1 to pushing more to TAGs. Perhaps each proposed project can be assigned to a TAG and a member of the TAG can lead a technical review and guide the project's leads on criteria for acceptance. That could help TOC reviews go more smoothly, make them more likely to succeed, and ensure projects and contributors don't get lost or feel unsupported along the journey.

It could also give first-time CNCF/TAG contributors an idea of where to start - they could pick an open project for the TAG and review and present it to the group.

As a start in this direction a couple months ago I created a label in TAG App Delivery to track project review requests from TOC: https://github.com/cncf/tag-app-delivery/issues?q=label%3Atoc-review

This is the official vote for the Environmental Conservation/Sustainability Working Group. 

Charter is here:  https://docs.google.com/document/d/1JaF7lSUmLQ2zmScmca6UF7PgbjMzSxjhhjx2LThThaY/edit?usp=sharing 

Please vote (+1/0/-1) by replying to this thread.

Remember that the TOC has binding votes only, but we do appreciate non-binding votes from the community as a sign of support! 

--

Amye Scavarda Perrin | Director of Developer Programs, CNCF | amye@...