On Tue, Sep 29, 2020 at 11:05 AM, Liz Rice wrote:

I think my wording could be better - the graduation requirement should be
for functional community control of the project roadmap (not merely a plan
for a community-controlled roadmap)

The start of your original email was clear, but the addition of the
word "plan" did make it a bit less so. Thank you for the clarification!

I've taken the liberty to re-format the thread below to inline replies
from earlier messages. I hope that the context doesn't change the
meaning of any quoted text from others.

On Tue, 29 Sep 2020 at 18:52, Alexis Richardson <alexis@...> wrote:

On Tue, Sep 29, 2020 at 10:48 AM, Bob Wise (AWS) wrote:

What would the mechanism used to be sure that a “plan” for a
community controlled roadmap becomes a reality? At present, holding
graduation as the milestone for that is, imho, one of the most
critical functions the CNCF has.

I (speaking only for myself) would like to understand the function
better. As I understand it, CNCF graduation is a signal that the project
has attained a certain level of "maturity". This is like a "Good
Housekeeping Seal of Approval" that gives users/customers a signal that
they can confidently adopt the software. Maintaining a high quality bar
for graduation is important in building the ongoing strong brand of the
CNCF, which I think plays a key role in supporting beneficial ecosystems
that are part of building and sustaining open-source software.

However, I currently view graduation milestones with the same kind of
skepticism of some engineering "qualification" practices I've seen,
where a product or service is very intensely tested before it is
released using tools and procedures that are not integrated into a
continuous automated testing system. "Qualification" around a major
milestone event gives you a snapshot-in-time view of quality, but modern
software changes more rapidly, so you need to be qualifying
continuously.

So too open-source projects need to change. Communities evolve.
Practices change and adapt. Communities that are not able to change and
evolve are more likely to become dysfunctional. Rather than obsessing
about if a vendor's interest is holding back features, I think you
should investigate how contentious decisions are made, and how disputes
are resolved among stakeholders. The roadmap/features is only one thing
that a development community, or other stakeholders, might disagree
about.

Liz clarified (above) that a proposed concern is to require functional
community control over the roadmap. I would shorten it: require a
functional community (in contrast to a dysfunctional community). I think
you (the CNCF TOC by applying the graduation seal-of-approval) need to
understand what community processes look like in practice, and if is a
healthy dynamic that aligns well with what we've learned so far about
building sustainable open-source software development communities and
commercial ecosystems. That'd be a hard job that I do not envy.

One tool that can help assess how well a community is functioning is a
free-form response template. But more deeply than how is it functioning,
a response template can help trigger, and demonstrate, more mindfulness
in community architecture. One I like is included in Python PIP-8002,
https://www.python.org/dev/peps/pep-8002/#annex-1-template-questions
I think that a well functioning community is one that has a process that
enables the evolution of its policies and practices over time. I like
that this template asks these questions:

3. How do you like the process?
* Which parts work well?
* Which parts could work better?
* When it doesn't work well, how does it look like?
* What would you change if it were only up to you?
[...]
5. Process evolution
* How did this process evolve historically?
* How can it be changed in the future?

The SC model could generate
* quarterly roadmap
* contributor ladder
* contingency plan

From my perspective, I care very little about any of these things
(though I may not understand what you actually mean). An open-source
license is my ultimate contingency plan. What I'm looking for otherwise
is a strong, welcoming, self-governing community, along with committed
companies participating in a growing commercial ecosystem around the
software.

Projects struggling to get maintainers when the owning entity is
unwilling to give up control to the community seems like a natural
consequence.

This mostly does not happen. Although due to some bad actors, it can.
Mostly projects are a handful of people on a mission, struggling to
make it work, in the face of massive demands and uncertainty.

Yes, there are a number of reasons why a project may not attract
contributors. There are a number of case studies where the overall
sentiment and perception was that an open-source project was too closely
held by a single commercial entity. Sometimes that has practical impacts
in the sustainability of the ecosystem that builds, maintains, and
supports the software.

In my opinion (biased from my experience and perspective), a key case
study in Linux Foundation history is the Xen project. We are fortunate
that the late Lars Kurth documented this case study well, see
http://slideshare.net/xen_com_mgr/lceu13-xen-project-lessons-learned.
When we formed the Xen Project as a Linux Foundation collaborative
project, there were a number of _symptoms_ that indicated that the
community, the project, and the ecosystem around it were trending toward
unhealthy. This was a sustainability concern for many stakeholders,
including me. Forming the Xen Project was only _part_ of addressing the
problem.

People who were most familiar with the Xen community were aware of some
the underlying causes of these symptoms, which were reflected in a
larger "image" problem for the project. Over many years the project and
its community gained a reputation of being difficult to work with,
inwardly focused, and not collaborative with other significant parts of
the open-source software ecosystem on which Xen was built--namely Linux
and QEMU, both of which were extensively modified and maintained as
separate "out of tree / not upstream" development branches for many
years. Some would call the software "forked".

At the time we formed the Xen Project at LF, it already had core
maintainers for multiple organizations, despite having earned a
reputation of being too closely held by Citrix. There are examples of
healthy projects with a small number of people from one organization
with "commit bit" access to the canonical repository, and also examples
of unhealthy projects that have people from multiple organizations with
"commit bit" access. So, I generally agree that the "multi-organization"
criterion is not a good one on its own. It _might_ be a symptom that
there's something else going on that deserves additional investigation.

So, TL;DR? Building a sustainable community is complex. The things that
may be the actual underlying risk factors in sustainability/longevity
may not be obvious in a surface level inspection. A plan is no guarantee
of future success. I think it takes applying mindful, intentional
community-building practices and a willingness to adapt.

--msw