1. Cncf-Toc
  2. Messages

Re: Sandbox process needs to evolve to support cross industry collaboation

Bob Killen
 

I agree on quite a few points :)  Replying in line with some thoughts

> We tried SIGs (now TAGs) doing due diligence for projects. The level
> of scrutiny, and the closeness to the guidance material available, was
> different across TAGs. In effect, this meant inconsistent processes
> which is arguably unfair. And in cases of disagreements, TOC is pulled
> in automatically anyway.

The TOC is the approval body and should be involved in DD, but I do think delegating portions of it to the TAGs is still a good idea and could play a large role in scaling the process. If there have been issues with varying levels of scrutiny in the past, this could be a mentorship and/or documentation opportunity. Think "ride-alongs" for reviewing DD, calling out what to look for, etc.  I also don't necessarily want to volunteer them, but TAG Contributor Strategy would be an excellent resource to pull in to review areas of governance and community health.

> What TAGs could provide is an initial proving ground, though: Projects
> could give a presentation and go through questions and feedback in a
> more limited scope, allowing them to polish their submittal.

+1 to involving them early, an initial consult would likely help with firming up applications before applying to Sandbox.

> While I know that the current sandbox process is designed to be very
> low barrier, I am still not convinced that this is an obviously
> desirable design goal. It is true that a neutral playing field is good
> and helps some projects grow. It is also true that "CNCF project"
> holds immense marketing value and many efforts are ephemeral, in
> particular if largely driven by perf & marketing.
> Back when sandbox criteria were relaxed, I was of the opinion that
> they should remain more stringent.

I have held the same opinion - I thought they should, to a degree, remain more stringent. While Sandbox does not have any formal marketing support from the CNCF, that doesn't mean companies or other groups can't market them as a "CNCF Project." Smaller or independent projects that might not have those sorts of resources will have a harder time climbing the ladder.

> I have come to wonder if four
> levels wouldn't be more appropriate: An initial runway on which
> projects can be put; but also pruned more aggressively if they do not
> show growth/adoption/the usual. E.g. once submitted they have three?
> six? twelve? months to show certain progress or are removed outright.

I was literally talking with a co-worker about this thought yesterday as a potential idea :)
I don't know if it's the answer, but I do really like the idea of a timebox with explicit criteria for exiting. It should not require a deep dive into the project to determine if they are ready to move up to sandbox. I'd also like to see restrictions on the branding/marketing of "CNCF Project" at this level. A potential alternative might be "Cloud Native Inception Project" or something along those lines.


> Another would be to rework the process & documentation; e.g.
> Incubation had distinct requirement docs which TAGs copied together
> and deduplicated back during the DD trials.

+1 to firming up requirements/docs. While I think there needs to be some room for TOC discretion, I think being more explicit with requirements will help reduce the toil involved with the DD process.


I have a slew more thoughts, but this subject might be a good discussion during a TOC meeting :)

- Bob



On Thu, May 5, 2022 at 7:38 AM Richard Hartmann <richih@...> wrote:
Replying top-level as my thoughts jump across the thread.


I didn't run the numbers, yet I believe that the pace of submissions
has picked up. That alone can increase backlog.

We tried SIGs (now TAGs) doing due diligence for projects. The level
of scrutiny, and the closeness to the guidance material available, was
different across TAGs. In effect, this meant inconsistent processes
which is arguably unfair. And in cases of disagreements, TOC is pulled
in automatically anyway.
A clear delegation from TOC might be possible, yet project advancement
is one of the main tasks of TOC and arguably what votees expect TOC to
do. In any case, it does change any of the underlying desires.

What TAGs could provide is an initial proving ground, though: Projects
could give a presentation and go through questions and feedback in a
more limited scope, allowing them to polish their submittal.


While I know that the current sandbox process is designed to be very
low barrier, I am still not convinced that this is an obviously
desirable design goal. It is true that a neutral playing field is good
and helps some projects grow. It is also true that "CNCF project"
holds immense marketing value and many efforts are ephemeral, in
particular if largely driven by perf & marketing.
Back when sandbox criteria were relaxed, I was of the opinion that
they should remain more stringent. I have come to wonder if four
levels wouldn't be more appropriate: An initial runway on which
projects can be put; but also pruned more aggressively if they do not
show growth/adoption/the usual. E.g. once submitted they have three?
six? twelve? months to show certain progress or are removed outright.
Medium term, this might also allow for a smaller jump towards
Incubating, which is currently significant.


Orthogonally, I believe we can manage expectations better. One
possible approach would be to create dashboards and reports of the
underlying data to help manage expectations and keep ourselves honest.
What are the average and median times a project takes from stage X to
stage Y? How has this changed over time?
Another would be to rework the process & documentation; e.g.
Incubation had distinct requirement docs which TAGs copied together
and deduplicated back during the DD trials.



Having seen things from both sides now, and since CNCF started, I can
understand both the frustrations about some timelines better and also
understand how a few dedicated people are trying to do their best with
the time they have. On all sides.


Best,
Richard
Join cncf-toc@lists.cncf.io to automatically receive all group messages.