[VOTE] k3s for Sandbox


Katie Gamanji
 

+1 binding

I would like to bring the end-user perspective to the discussion. At this stage, there are different requirements to operate a k8s cluster and some teams and companies are using k3s (due to its opinionated structure) as a solid exploration point. It benefits the community as a project and I CNCF will be a great home to further the growth of this project. 

Katie Gamanji 

On Mon, Aug 10, 2020 at 9:27 PM Bob Wise (AWS) via lists.cncf.io <wisebob=amazon.com@...> wrote:

The chasm statement is interesting in the abstract. The well-deserved implication is “production use” is far to broad a term for a huge variety of situations, and I withdraw the question. Has the project made it clear how and when security issues (both under embargo and not) that are upstream will be handled, or not? That way the users can make their own judgements about whether that is acceptable risk for whatever their usage scenario is.

 

Conformance is a set of test cases. You don’t need to (nor should you) have to have any set of upstream code to pass conformance. This is good and as it should be. It can be both true that a project is conformant, and be either a fork or a complete re implementation. We tripped over this way back when during the Openstack days, and I think the Kubernetes project took the lesson. It is also possible to configure a non-forked/upstream distro to not pass conformance.

 

Consequently, statements of the form, “x is not a fork because it is conformant” are misleading.

 

Forks represent a compatibility risk for users. Again, the users are the ones that need to evaluate the risk and decide how fully they believe conformance covers their intended usage. We should be clear with the communities and users so they are able to make their own judgements.

 

To your specific comment: It’s not about whether etcd is present. That’s not what I was saying. It’s about whether the code that is needed to replace etcd with something else is a fork.

 

-bw

 

From: Matt Farina <matt@...>
Date: Monday, August 10, 2020 at 12:21 PM
To: "Wise, Bob" <wisebob@...>, CNCF TOC <cncf-toc@...>
Subject: RE: [EXTERNAL] [cncf-toc] [VOTE] k3s for Sandbox

 

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.

 

I can speak to some of this...

 

Are we clear about sandbox projects as “not recommended for production”

 

The CNCF describes projects in terms of crossing the chasm. You can find that here. Sandbox is labeled as being for the innovators.

 

A production recommendation really depends on who you are. An enterprise is different from an early stage startup. What one would suggest be used in prod for a new startup or for The Home Depot is going to be very different. I like the description in terms of the technology adoption lifecycle instead.

 

The forking issue, e.g. not using etcd. I believe there are other not-packaging-or-tooling related changes.

 

This brings up a good question that, I think, is a bit more complicated. What parts of Kubernetes are required to be present to be conformant? I would leave this up to the conformance folks. For example, would the use of Virtual Kubelet (another sandbox project) be a change to mean to far to allow? Why?

 

From the perspective of someone deploying workloads into a cluster I don't see how it matters if the API and environment is conformant for me to run workloads in.

 

If I'm missing something I would be curious to know.

 

For clarity, the idea of upstream supporting multiple persistence backends in addition to etcd is a great idea – I would love to see that upstream.

 

This idea isn't new. It has been brought up before. While I can't state where this sits today I do know that bringing in new ideas to k8s can be difficult now because of people working in certain ways to maintain them. In the past SIG Architecture would tell people to innovate things out of tree. If it was a good idea and proved to be useful then K8s would potentially look to bring it back in house.

 

I like this idea. But, I think it needs to be proved out of tree before it can be brought in.

 

On Mon, Aug 10, 2020, at 1:48 PM, Wise, Bob wrote:

To me, this does not feel like either of those two upstream projects. Characterizing k3s as packaging only or tooling used to manage Kubernetes is not addressing:

 

  1. The forking issue, e.g. not using etcd. I believe there are other not-packaging-or-tooling related changes.
  2. The precedent setting for both distros and forks.
  3. Security posture w.r.t. to the PSC and embargo issue handling. Not arguably relevant if the project stays in sandbox, but relevant to all those that follow and critical if the path is incubation-bound. Are we clear about sandbox projects as “not recommended for production”?

 

Was there a voted-on statement from the Kubernetes Steering committee on this? Were the statements made more of the “we aren’t the TOC so we don’t have standing to object”, or “wow this is awesome and we support it”?

 

For clarity, the idea of upstream supporting multiple persistence backends in addition to etcd is a great idea – I would love to see that upstream.

 

-Bob

 

From: <cncf-toc@...> on behalf of Matt Farina <matt@...>
Date: Monday, August 10, 2020 at 10:05 AM
To: CNCF TOC <cncf-toc@...>
Subject: RE: [EXTERNAL] [cncf-toc] [VOTE] k3s for Sandbox

 

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.

 

I expect there are many people in this thread that lack some context. So, I figured I'd try to add some. If someone has more to add or a correction please do so. These are intermixed with some of my personal opinions, of course.

 

k3s feels very similar to Kind (https://github.com/kubernetes-sigs/kind) and minikube (https://github.com/kubernetes/minikube) both of which are Kubernetes sub-projects.

 

There are two things about this that jump out to me.

 

First, kind, k3s, and minikube are different in ways that are important to those who use them. Those subtle differences matter. This happens with other distros, too.

 

kind was developed with Kubernetes test infrastructure automation in mind. In fact, the docs say "kind was primarily designed for testing Kubernetes itself, but may be used for local development or CI.". Minikube, which has been around a lot longer, targets Kubernetes application developers. Their docs note, "minikube's primary goals are to be the best tool for local Kubernetes application development and to support all Kubernetes features that fit."

 

This distinction is important as we see people performing different roles. Tools for those different roles will likely look different in the end.

 

k3s targets a different group from these other two. Their docs note, "The certified Kubernetes distribution built for IoT & Edge computing". This is a different situation from the other two and I would expect the experiencing of using it to look different.

 

It may feel the same on the surface but the subtle goals are going to lead it in some different directions. Those are good as one size does not fit all.

 

Second, while minikube and kind are k8s sub-projects that doesn't mean all distros that fall under the CNCF need to be. kind you would expect to be be part of the kubernetes project because it was developed with testing kubernetes itself in mind. If minikube were started today it may have been something entirely separate from the kubernetes project. Over the years we have had discussions and debates on this theoretical topic.

 

The idea that all of these things must or should be part of the Kubernetes project doesn't fit with the discussions the k8s community has had over the years. In fact, people come to Kubernetes with ideas for changes and we regularly tell them to do them as part a different project rather than in k8s.

 

I'm surprised to see a push for distros that are open source to be part of the k8s project if they are to be in the CNCF.

 

I would like to to see k3s follow the same approach as those projects and apply to become a k8s sub-project first, rather than a standalone CNCF sandbox project.

This would ensure consistency, and give me confidence that Kubernetes experts have reviewed the project. Even if the k8s community ultimately says no, it wouldn't mean automatic no for a stand alone CNCF project, but would provide valuable insight for a CNCF application.

 

I would prefer to see diversity over consistency. If a distribution is conformant (we have tests for that) there should be room for diversity in the way things are done. Sometimes this will be experiments. Sometimes that will mean one distro is not consistent with another. For example, some distros ship with CRDs pre-installed which means an extended API. Sometimes that will mean someone swapped out a component with another version that uses the same APIs.

 

The sandbox is a great please to experiment in a cross company way.

 

The k8s community is amazingly busy. The people are busy. I wouldn't want to put more on their plates but rather take it off. Enabling groups to operate autonomously from each other or in a loosely coupled manner helps with that. Sub-projects of Kubernetes are limited in their ability to do that. A product of Kubernetes trying to efficiently handle the scale of people and decisions.

 

Being a separate CNCF project enables a loose coupling. It doesn't add work to an overworked k8s community but enables collaboration. This is an often overlooked element.

 

Open source projects aren't companies that fall along nice divisional lines. They are far more organic while variance in people, process, and ideas flourish. A fertile place for that is important.

 

On Mon, Aug 3, 2020, at 5:06 AM, Saad Ali via lists.cncf.io wrote:

Abstain

 

Unfortunately I missed the "Joint CNCF TOC/Kubernetes Steering Committee" meeting. But I talked to some folks afterwards to fill me in.

 

Overall, I'd like to avoid setting precedent that a project unable to generate consensus within the Kubernetes community, can bypass the community by going straight to the CNCF.

While that may not be what happened here, k3s feels very similar to Kind (https://github.com/kubernetes-sigs/kind) and minikube (https://github.com/kubernetes/minikube) both of which are Kubernetes sub-projects.

I would like to to see k3s follow the same approach as those projects and apply to become a k8s sub-project first, rather than a standalone CNCF sandbox project.

This would ensure consistency, and give me confidence that Kubernetes experts have reviewed the project. Even if the k8s community ultimately says no, it wouldn't mean automatic no for a stand alone CNCF project, but would provide valuable insight for a CNCF application.

While I realize that building technical consensus is hard, especially in a project as large as Kubernetes, I believe doing so is critical for healthy communities.

 

Why not vote "-1"? The revised CNCF Sandbox project guidelines lower the bar for sandbox, making it a *tool* that can be used by anyone who needs a neutral place to host IP and collaborate on new projects with minimal overhead rather then a stepping stone towards incubation/graduation. This lets the TOC and CNCF SIGs to be more discriminating for incubation/graduation projects (only accept projects at that level that "makes sense in the CNCF ecosystem") while allowing the sandbox to serve as a test bed for innovation.

 

 


Bob Wise (AWS)
 

The chasm statement is interesting in the abstract. The well-deserved implication is “production use” is far to broad a term for a huge variety of situations, and I withdraw the question. Has the project made it clear how and when security issues (both under embargo and not) that are upstream will be handled, or not? That way the users can make their own judgements about whether that is acceptable risk for whatever their usage scenario is.

 

Conformance is a set of test cases. You don’t need to (nor should you) have to have any set of upstream code to pass conformance. This is good and as it should be. It can be both true that a project is conformant, and be either a fork or a complete re implementation. We tripped over this way back when during the Openstack days, and I think the Kubernetes project took the lesson. It is also possible to configure a non-forked/upstream distro to not pass conformance.

 

Consequently, statements of the form, “x is not a fork because it is conformant” are misleading.

 

Forks represent a compatibility risk for users. Again, the users are the ones that need to evaluate the risk and decide how fully they believe conformance covers their intended usage. We should be clear with the communities and users so they are able to make their own judgements.

 

To your specific comment: It’s not about whether etcd is present. That’s not what I was saying. It’s about whether the code that is needed to replace etcd with something else is a fork.

 

-bw

 

From: Matt Farina <matt@...>
Date: Monday, August 10, 2020 at 12:21 PM
To: "Wise, Bob" <wisebob@...>, CNCF TOC <cncf-toc@...>
Subject: RE: [EXTERNAL] [cncf-toc] [VOTE] k3s for Sandbox

 

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.

 

I can speak to some of this...

 

Are we clear about sandbox projects as “not recommended for production”

 

The CNCF describes projects in terms of crossing the chasm. You can find that here. Sandbox is labeled as being for the innovators.

 

A production recommendation really depends on who you are. An enterprise is different from an early stage startup. What one would suggest be used in prod for a new startup or for The Home Depot is going to be very different. I like the description in terms of the technology adoption lifecycle instead.

 

The forking issue, e.g. not using etcd. I believe there are other not-packaging-or-tooling related changes.

 

This brings up a good question that, I think, is a bit more complicated. What parts of Kubernetes are required to be present to be conformant? I would leave this up to the conformance folks. For example, would the use of Virtual Kubelet (another sandbox project) be a change to mean to far to allow? Why?

 

From the perspective of someone deploying workloads into a cluster I don't see how it matters if the API and environment is conformant for me to run workloads in.

 

If I'm missing something I would be curious to know.

 

For clarity, the idea of upstream supporting multiple persistence backends in addition to etcd is a great idea – I would love to see that upstream.

 

This idea isn't new. It has been brought up before. While I can't state where this sits today I do know that bringing in new ideas to k8s can be difficult now because of people working in certain ways to maintain them. In the past SIG Architecture would tell people to innovate things out of tree. If it was a good idea and proved to be useful then K8s would potentially look to bring it back in house.

 

I like this idea. But, I think it needs to be proved out of tree before it can be brought in.

 

On Mon, Aug 10, 2020, at 1:48 PM, Wise, Bob wrote:

To me, this does not feel like either of those two upstream projects. Characterizing k3s as packaging only or tooling used to manage Kubernetes is not addressing:

 

  1. The forking issue, e.g. not using etcd. I believe there are other not-packaging-or-tooling related changes.
  2. The precedent setting for both distros and forks.
  3. Security posture w.r.t. to the PSC and embargo issue handling. Not arguably relevant if the project stays in sandbox, but relevant to all those that follow and critical if the path is incubation-bound. Are we clear about sandbox projects as “not recommended for production”?

 

Was there a voted-on statement from the Kubernetes Steering committee on this? Were the statements made more of the “we aren’t the TOC so we don’t have standing to object”, or “wow this is awesome and we support it”?

 

For clarity, the idea of upstream supporting multiple persistence backends in addition to etcd is a great idea – I would love to see that upstream.

 

-Bob

 

From: <cncf-toc@...> on behalf of Matt Farina <matt@...>
Date: Monday, August 10, 2020 at 10:05 AM
To: CNCF TOC <cncf-toc@...>
Subject: RE: [EXTERNAL] [cncf-toc] [VOTE] k3s for Sandbox

 

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.

 

I expect there are many people in this thread that lack some context. So, I figured I'd try to add some. If someone has more to add or a correction please do so. These are intermixed with some of my personal opinions, of course.

 

k3s feels very similar to Kind (https://github.com/kubernetes-sigs/kind) and minikube (https://github.com/kubernetes/minikube) both of which are Kubernetes sub-projects.

 

There are two things about this that jump out to me.

 

First, kind, k3s, and minikube are different in ways that are important to those who use them. Those subtle differences matter. This happens with other distros, too.

 

kind was developed with Kubernetes test infrastructure automation in mind. In fact, the docs say "kind was primarily designed for testing Kubernetes itself, but may be used for local development or CI.". Minikube, which has been around a lot longer, targets Kubernetes application developers. Their docs note, "minikube's primary goals are to be the best tool for local Kubernetes application development and to support all Kubernetes features that fit."

 

This distinction is important as we see people performing different roles. Tools for those different roles will likely look different in the end.

 

k3s targets a different group from these other two. Their docs note, "The certified Kubernetes distribution built for IoT & Edge computing". This is a different situation from the other two and I would expect the experiencing of using it to look different.

 

It may feel the same on the surface but the subtle goals are going to lead it in some different directions. Those are good as one size does not fit all.

 

Second, while minikube and kind are k8s sub-projects that doesn't mean all distros that fall under the CNCF need to be. kind you would expect to be be part of the kubernetes project because it was developed with testing kubernetes itself in mind. If minikube were started today it may have been something entirely separate from the kubernetes project. Over the years we have had discussions and debates on this theoretical topic.

 

The idea that all of these things must or should be part of the Kubernetes project doesn't fit with the discussions the k8s community has had over the years. In fact, people come to Kubernetes with ideas for changes and we regularly tell them to do them as part a different project rather than in k8s.

 

I'm surprised to see a push for distros that are open source to be part of the k8s project if they are to be in the CNCF.

 

I would like to to see k3s follow the same approach as those projects and apply to become a k8s sub-project first, rather than a standalone CNCF sandbox project.

This would ensure consistency, and give me confidence that Kubernetes experts have reviewed the project. Even if the k8s community ultimately says no, it wouldn't mean automatic no for a stand alone CNCF project, but would provide valuable insight for a CNCF application.

 

I would prefer to see diversity over consistency. If a distribution is conformant (we have tests for that) there should be room for diversity in the way things are done. Sometimes this will be experiments. Sometimes that will mean one distro is not consistent with another. For example, some distros ship with CRDs pre-installed which means an extended API. Sometimes that will mean someone swapped out a component with another version that uses the same APIs.

 

The sandbox is a great please to experiment in a cross company way.

 

The k8s community is amazingly busy. The people are busy. I wouldn't want to put more on their plates but rather take it off. Enabling groups to operate autonomously from each other or in a loosely coupled manner helps with that. Sub-projects of Kubernetes are limited in their ability to do that. A product of Kubernetes trying to efficiently handle the scale of people and decisions.

 

Being a separate CNCF project enables a loose coupling. It doesn't add work to an overworked k8s community but enables collaboration. This is an often overlooked element.

 

Open source projects aren't companies that fall along nice divisional lines. They are far more organic while variance in people, process, and ideas flourish. A fertile place for that is important.

 

On Mon, Aug 3, 2020, at 5:06 AM, Saad Ali via lists.cncf.io wrote:

Abstain

 

Unfortunately I missed the "Joint CNCF TOC/Kubernetes Steering Committee" meeting. But I talked to some folks afterwards to fill me in.

 

Overall, I'd like to avoid setting precedent that a project unable to generate consensus within the Kubernetes community, can bypass the community by going straight to the CNCF.

While that may not be what happened here, k3s feels very similar to Kind (https://github.com/kubernetes-sigs/kind) and minikube (https://github.com/kubernetes/minikube) both of which are Kubernetes sub-projects.

I would like to to see k3s follow the same approach as those projects and apply to become a k8s sub-project first, rather than a standalone CNCF sandbox project.

This would ensure consistency, and give me confidence that Kubernetes experts have reviewed the project. Even if the k8s community ultimately says no, it wouldn't mean automatic no for a stand alone CNCF project, but would provide valuable insight for a CNCF application.

While I realize that building technical consensus is hard, especially in a project as large as Kubernetes, I believe doing so is critical for healthy communities.

 

Why not vote "-1"? The revised CNCF Sandbox project guidelines lower the bar for sandbox, making it a *tool* that can be used by anyone who needs a neutral place to host IP and collaborate on new projects with minimal overhead rather then a stepping stone towards incubation/graduation. This lets the TOC and CNCF SIGs to be more discriminating for incubation/graduation projects (only accept projects at that level that "makes sense in the CNCF ecosystem") while allowing the sandbox to serve as a test bed for innovation.

 

 


Matt Farina
 

I can speak to some of this...

Are we clear about sandbox projects as “not recommended for production”

The CNCF describes projects in terms of crossing the chasm. You can find that here. Sandbox is labeled as being for the innovators.

A production recommendation really depends on who you are. An enterprise is different from an early stage startup. What one would suggest be used in prod for a new startup or for The Home Depot is going to be very different. I like the description in terms of the technology adoption lifecycle instead.

The forking issue, e.g. not using etcd. I believe there are other not-packaging-or-tooling related changes.

This brings up a good question that, I think, is a bit more complicated. What parts of Kubernetes are required to be present to be conformant? I would leave this up to the conformance folks. For example, would the use of Virtual Kubelet (another sandbox project) be a change to mean to far to allow? Why?

From the perspective of someone deploying workloads into a cluster I don't see how it matters if the API and environment is conformant for me to run workloads in.

If I'm missing something I would be curious to know.

For clarity, the idea of upstream supporting multiple persistence backends in addition to etcd is a great idea – I would love to see that upstream.

This idea isn't new. It has been brought up before. While I can't state where this sits today I do know that bringing in new ideas to k8s can be difficult now because of people working in certain ways to maintain them. In the past SIG Architecture would tell people to innovate things out of tree. If it was a good idea and proved to be useful then K8s would potentially look to bring it back in house.

I like this idea. But, I think it needs to be proved out of tree before it can be brought in.

On Mon, Aug 10, 2020, at 1:48 PM, Wise, Bob wrote:

To me, this does not feel like either of those two upstream projects. Characterizing k3s as packaging only or tooling used to manage Kubernetes is not addressing:

 

  1. The forking issue, e.g. not using etcd. I believe there are other not-packaging-or-tooling related changes.
  2. The precedent setting for both distros and forks.
  3. Security posture w.r.t. to the PSC and embargo issue handling. Not arguably relevant if the project stays in sandbox, but relevant to all those that follow and critical if the path is incubation-bound. Are we clear about sandbox projects as “not recommended for production”?

 

Was there a voted-on statement from the Kubernetes Steering committee on this? Were the statements made more of the “we aren’t the TOC so we don’t have standing to object”, or “wow this is awesome and we support it”?

 

For clarity, the idea of upstream supporting multiple persistence backends in addition to etcd is a great idea – I would love to see that upstream.

 

-Bob

 

From: <cncf-toc@...> on behalf of Matt Farina <matt@...>
Date: Monday, August 10, 2020 at 10:05 AM
To: CNCF TOC <cncf-toc@...>
Subject: RE: [EXTERNAL] [cncf-toc] [VOTE] k3s for Sandbox

 

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.

 

I expect there are many people in this thread that lack some context. So, I figured I'd try to add some. If someone has more to add or a correction please do so. These are intermixed with some of my personal opinions, of course.

 

k3s feels very similar to Kind (https://github.com/kubernetes-sigs/kind) and minikube (https://github.com/kubernetes/minikube) both of which are Kubernetes sub-projects.

 

There are two things about this that jump out to me.

 

First, kind, k3s, and minikube are different in ways that are important to those who use them. Those subtle differences matter. This happens with other distros, too.

 

kind was developed with Kubernetes test infrastructure automation in mind. In fact, the docs say "kind was primarily designed for testing Kubernetes itself, but may be used for local development or CI.". Minikube, which has been around a lot longer, targets Kubernetes application developers. Their docs note, "minikube's primary goals are to be the best tool for local Kubernetes application development and to support all Kubernetes features that fit."

 

This distinction is important as we see people performing different roles. Tools for those different roles will likely look different in the end.

 

k3s targets a different group from these other two. Their docs note, "The certified Kubernetes distribution built for IoT & Edge computing". This is a different situation from the other two and I would expect the experiencing of using it to look different.

 

It may feel the same on the surface but the subtle goals are going to lead it in some different directions. Those are good as one size does not fit all.

 

Second, while minikube and kind are k8s sub-projects that doesn't mean all distros that fall under the CNCF need to be. kind you would expect to be be part of the kubernetes project because it was developed with testing kubernetes itself in mind. If minikube were started today it may have been something entirely separate from the kubernetes project. Over the years we have had discussions and debates on this theoretical topic.

 

The idea that all of these things must or should be part of the Kubernetes project doesn't fit with the discussions the k8s community has had over the years. In fact, people come to Kubernetes with ideas for changes and we regularly tell them to do them as part a different project rather than in k8s.

 

I'm surprised to see a push for distros that are open source to be part of the k8s project if they are to be in the CNCF.

 

I would like to to see k3s follow the same approach as those projects and apply to become a k8s sub-project first, rather than a standalone CNCF sandbox project.

This would ensure consistency, and give me confidence that Kubernetes experts have reviewed the project. Even if the k8s community ultimately says no, it wouldn't mean automatic no for a stand alone CNCF project, but would provide valuable insight for a CNCF application.

 

I would prefer to see diversity over consistency. If a distribution is conformant (we have tests for that) there should be room for diversity in the way things are done. Sometimes this will be experiments. Sometimes that will mean one distro is not consistent with another. For example, some distros ship with CRDs pre-installed which means an extended API. Sometimes that will mean someone swapped out a component with another version that uses the same APIs.

 

The sandbox is a great please to experiment in a cross company way.

 

The k8s community is amazingly busy. The people are busy. I wouldn't want to put more on their plates but rather take it off. Enabling groups to operate autonomously from each other or in a loosely coupled manner helps with that. Sub-projects of Kubernetes are limited in their ability to do that. A product of Kubernetes trying to efficiently handle the scale of people and decisions.

 

Being a separate CNCF project enables a loose coupling. It doesn't add work to an overworked k8s community but enables collaboration. This is an often overlooked element.

 

Open source projects aren't companies that fall along nice divisional lines. They are far more organic while variance in people, process, and ideas flourish. A fertile place for that is important.

 

On Mon, Aug 3, 2020, at 5:06 AM, Saad Ali via lists.cncf.io wrote:

Abstain

 

Unfortunately I missed the "Joint CNCF TOC/Kubernetes Steering Committee" meeting. But I talked to some folks afterwards to fill me in.

 

Overall, I'd like to avoid setting precedent that a project unable to generate consensus within the Kubernetes community, can bypass the community by going straight to the CNCF.

While that may not be what happened here, k3s feels very similar to Kind (https://github.com/kubernetes-sigs/kind) and minikube (https://github.com/kubernetes/minikube) both of which are Kubernetes sub-projects.

I would like to to see k3s follow the same approach as those projects and apply to become a k8s sub-project first, rather than a standalone CNCF sandbox project.

This would ensure consistency, and give me confidence that Kubernetes experts have reviewed the project. Even if the k8s community ultimately says no, it wouldn't mean automatic no for a stand alone CNCF project, but would provide valuable insight for a CNCF application.

While I realize that building technical consensus is hard, especially in a project as large as Kubernetes, I believe doing so is critical for healthy communities.

 

Why not vote "-1"? The revised CNCF Sandbox project guidelines lower the bar for sandbox, making it a *tool* that can be used by anyone who needs a neutral place to host IP and collaborate on new projects with minimal overhead rather then a stepping stone towards incubation/graduation. This lets the TOC and CNCF SIGs to be more discriminating for incubation/graduation projects (only accept projects at that level that "makes sense in the CNCF ecosystem") while allowing the sandbox to serve as a test bed for innovation.

 




Liz Rice
 

sandbox projects are early stage / experimental and where we want to enable neutral collaboration. There is no DD so we are not giving any recommendations (Including not recommending for production). There are no security requirements for Sandbox projects.

KSC were unequivocal about not supporting k3s as a subproject at this time, with reasons including not proliferating the number of release artifacts that k8s folks have to manage & test. That doesn't mean it's a closed door forever.

But k3s wants a neutral collaboration space for something that is clearly cloud native and an experiment with a lot of community support. I don't see any good reason not to encourage that.

Although we prefer to be consistent TOC has discretion, so accepting k3s does not force our hand for accepting future "forks" or "distros"



On Mon, 10 Aug 2020 at 18:56, Bob Wise (AWS) via lists.cncf.io <wisebob=amazon.com@...> wrote:

To me, this does not feel like either of those two upstream projects. Characterizing k3s as packaging only or tooling used to manage Kubernetes is not addressing:

 

  1. The forking issue, e.g. not using etcd. I believe there are other not-packaging-or-tooling related changes.
  2. The precedent setting for both distros and forks.
  3. Security posture w.r.t. to the PSC and embargo issue handling. Not arguably relevant if the project stays in sandbox, but relevant to all those that follow and critical if the path is incubation-bound. Are we clear about sandbox projects as “not recommended for production”?

 

Was there a voted-on statement from the Kubernetes Steering committee on this? Were the statements made more of the “we aren’t the TOC so we don’t have standing to object”, or “wow this is awesome and we support it”?

 

For clarity, the idea of upstream supporting multiple persistence backends in addition to etcd is a great idea – I would love to see that upstream.

 

-Bob

 

From: <cncf-toc@...> on behalf of Matt Farina <matt@...>
Date: Monday, August 10, 2020 at 10:05 AM
To: CNCF TOC <cncf-toc@...>
Subject: RE: [EXTERNAL] [cncf-toc] [VOTE] k3s for Sandbox

 

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.

 

I expect there are many people in this thread that lack some context. So, I figured I'd try to add some. If someone has more to add or a correction please do so. These are intermixed with some of my personal opinions, of course.

 

k3s feels very similar to Kind (https://github.com/kubernetes-sigs/kind) and minikube (https://github.com/kubernetes/minikube) both of which are Kubernetes sub-projects.

 

There are two things about this that jump out to me.

 

First, kind, k3s, and minikube are different in ways that are important to those who use them. Those subtle differences matter. This happens with other distros, too.

 

kind was developed with Kubernetes test infrastructure automation in mind. In fact, the docs say "kind was primarily designed for testing Kubernetes itself, but may be used for local development or CI.". Minikube, which has been around a lot longer, targets Kubernetes application developers. Their docs note, "minikube's primary goals are to be the best tool for local Kubernetes application development and to support all Kubernetes features that fit."

 

This distinction is important as we see people performing different roles. Tools for those different roles will likely look different in the end.

 

k3s targets a different group from these other two. Their docs note, "The certified Kubernetes distribution built for IoT & Edge computing". This is a different situation from the other two and I would expect the experiencing of using it to look different.

 

It may feel the same on the surface but the subtle goals are going to lead it in some different directions. Those are good as one size does not fit all.

 

Second, while minikube and kind are k8s sub-projects that doesn't mean all distros that fall under the CNCF need to be. kind you would expect to be be part of the kubernetes project because it was developed with testing kubernetes itself in mind. If minikube were started today it may have been something entirely separate from the kubernetes project. Over the years we have had discussions and debates on this theoretical topic.

 

The idea that all of these things must or should be part of the Kubernetes project doesn't fit with the discussions the k8s community has had over the years. In fact, people come to Kubernetes with ideas for changes and we regularly tell them to do them as part a different project rather than in k8s.

 

I'm surprised to see a push for distros that are open source to be part of the k8s project if they are to be in the CNCF.

 

I would like to to see k3s follow the same approach as those projects and apply to become a k8s sub-project first, rather than a standalone CNCF sandbox project.

This would ensure consistency, and give me confidence that Kubernetes experts have reviewed the project. Even if the k8s community ultimately says no, it wouldn't mean automatic no for a stand alone CNCF project, but would provide valuable insight for a CNCF application.

 

I would prefer to see diversity over consistency. If a distribution is conformant (we have tests for that) there should be room for diversity in the way things are done. Sometimes this will be experiments. Sometimes that will mean one distro is not consistent with another. For example, some distros ship with CRDs pre-installed which means an extended API. Sometimes that will mean someone swapped out a component with another version that uses the same APIs.

 

The sandbox is a great please to experiment in a cross company way.

 

The k8s community is amazingly busy. The people are busy. I wouldn't want to put more on their plates but rather take it off. Enabling groups to operate autonomously from each other or in a loosely coupled manner helps with that. Sub-projects of Kubernetes are limited in their ability to do that. A product of Kubernetes trying to efficiently handle the scale of people and decisions.

 

Being a separate CNCF project enables a loose coupling. It doesn't add work to an overworked k8s community but enables collaboration. This is an often overlooked element.

 

Open source projects aren't companies that fall along nice divisional lines. They are far more organic while variance in people, process, and ideas flourish. A fertile place for that is important.

 

On Mon, Aug 3, 2020, at 5:06 AM, Saad Ali via lists.cncf.io wrote:

Abstain

 

Unfortunately I missed the "Joint CNCF TOC/Kubernetes Steering Committee" meeting. But I talked to some folks afterwards to fill me in.

 

Overall, I'd like to avoid setting precedent that a project unable to generate consensus within the Kubernetes community, can bypass the community by going straight to the CNCF.

While that may not be what happened here, k3s feels very similar to Kind (https://github.com/kubernetes-sigs/kind) and minikube (https://github.com/kubernetes/minikube) both of which are Kubernetes sub-projects.

I would like to to see k3s follow the same approach as those projects and apply to become a k8s sub-project first, rather than a standalone CNCF sandbox project.

This would ensure consistency, and give me confidence that Kubernetes experts have reviewed the project. Even if the k8s community ultimately says no, it wouldn't mean automatic no for a stand alone CNCF project, but would provide valuable insight for a CNCF application.

While I realize that building technical consensus is hard, especially in a project as large as Kubernetes, I believe doing so is critical for healthy communities.

 

Why not vote "-1"? The revised CNCF Sandbox project guidelines lower the bar for sandbox, making it a *tool* that can be used by anyone who needs a neutral place to host IP and collaborate on new projects with minimal overhead rather then a stepping stone towards incubation/graduation. This lets the TOC and CNCF SIGs to be more discriminating for incubation/graduation projects (only accept projects at that level that "makes sense in the CNCF ecosystem") while allowing the sandbox to serve as a test bed for innovation.

 


Bob Wise (AWS)
 

To me, this does not feel like either of those two upstream projects. Characterizing k3s as packaging only or tooling used to manage Kubernetes is not addressing:

 

  1. The forking issue, e.g. not using etcd. I believe there are other not-packaging-or-tooling related changes.
  2. The precedent setting for both distros and forks.
  3. Security posture w.r.t. to the PSC and embargo issue handling. Not arguably relevant if the project stays in sandbox, but relevant to all those that follow and critical if the path is incubation-bound. Are we clear about sandbox projects as “not recommended for production”?

 

Was there a voted-on statement from the Kubernetes Steering committee on this? Were the statements made more of the “we aren’t the TOC so we don’t have standing to object”, or “wow this is awesome and we support it”?

 

For clarity, the idea of upstream supporting multiple persistence backends in addition to etcd is a great idea – I would love to see that upstream.

 

-Bob

 

From: <cncf-toc@...> on behalf of Matt Farina <matt@...>
Date: Monday, August 10, 2020 at 10:05 AM
To: CNCF TOC <cncf-toc@...>
Subject: RE: [EXTERNAL] [cncf-toc] [VOTE] k3s for Sandbox

 

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.

 

I expect there are many people in this thread that lack some context. So, I figured I'd try to add some. If someone has more to add or a correction please do so. These are intermixed with some of my personal opinions, of course.

 

k3s feels very similar to Kind (https://github.com/kubernetes-sigs/kind) and minikube (https://github.com/kubernetes/minikube) both of which are Kubernetes sub-projects.

 

There are two things about this that jump out to me.

 

First, kind, k3s, and minikube are different in ways that are important to those who use them. Those subtle differences matter. This happens with other distros, too.

 

kind was developed with Kubernetes test infrastructure automation in mind. In fact, the docs say "kind was primarily designed for testing Kubernetes itself, but may be used for local development or CI.". Minikube, which has been around a lot longer, targets Kubernetes application developers. Their docs note, "minikube's primary goals are to be the best tool for local Kubernetes application development and to support all Kubernetes features that fit."

 

This distinction is important as we see people performing different roles. Tools for those different roles will likely look different in the end.

 

k3s targets a different group from these other two. Their docs note, "The certified Kubernetes distribution built for IoT & Edge computing". This is a different situation from the other two and I would expect the experiencing of using it to look different.

 

It may feel the same on the surface but the subtle goals are going to lead it in some different directions. Those are good as one size does not fit all.

 

Second, while minikube and kind are k8s sub-projects that doesn't mean all distros that fall under the CNCF need to be. kind you would expect to be be part of the kubernetes project because it was developed with testing kubernetes itself in mind. If minikube were started today it may have been something entirely separate from the kubernetes project. Over the years we have had discussions and debates on this theoretical topic.

 

The idea that all of these things must or should be part of the Kubernetes project doesn't fit with the discussions the k8s community has had over the years. In fact, people come to Kubernetes with ideas for changes and we regularly tell them to do them as part a different project rather than in k8s.

 

I'm surprised to see a push for distros that are open source to be part of the k8s project if they are to be in the CNCF.

 

I would like to to see k3s follow the same approach as those projects and apply to become a k8s sub-project first, rather than a standalone CNCF sandbox project.

This would ensure consistency, and give me confidence that Kubernetes experts have reviewed the project. Even if the k8s community ultimately says no, it wouldn't mean automatic no for a stand alone CNCF project, but would provide valuable insight for a CNCF application.

 

I would prefer to see diversity over consistency. If a distribution is conformant (we have tests for that) there should be room for diversity in the way things are done. Sometimes this will be experiments. Sometimes that will mean one distro is not consistent with another. For example, some distros ship with CRDs pre-installed which means an extended API. Sometimes that will mean someone swapped out a component with another version that uses the same APIs.

 

The sandbox is a great please to experiment in a cross company way.

 

The k8s community is amazingly busy. The people are busy. I wouldn't want to put more on their plates but rather take it off. Enabling groups to operate autonomously from each other or in a loosely coupled manner helps with that. Sub-projects of Kubernetes are limited in their ability to do that. A product of Kubernetes trying to efficiently handle the scale of people and decisions.

 

Being a separate CNCF project enables a loose coupling. It doesn't add work to an overworked k8s community but enables collaboration. This is an often overlooked element.

 

Open source projects aren't companies that fall along nice divisional lines. They are far more organic while variance in people, process, and ideas flourish. A fertile place for that is important.

 

On Mon, Aug 3, 2020, at 5:06 AM, Saad Ali via lists.cncf.io wrote:

Abstain

 

Unfortunately I missed the "Joint CNCF TOC/Kubernetes Steering Committee" meeting. But I talked to some folks afterwards to fill me in.

 

Overall, I'd like to avoid setting precedent that a project unable to generate consensus within the Kubernetes community, can bypass the community by going straight to the CNCF.

While that may not be what happened here, k3s feels very similar to Kind (https://github.com/kubernetes-sigs/kind) and minikube (https://github.com/kubernetes/minikube) both of which are Kubernetes sub-projects.

I would like to to see k3s follow the same approach as those projects and apply to become a k8s sub-project first, rather than a standalone CNCF sandbox project.

This would ensure consistency, and give me confidence that Kubernetes experts have reviewed the project. Even if the k8s community ultimately says no, it wouldn't mean automatic no for a stand alone CNCF project, but would provide valuable insight for a CNCF application.

While I realize that building technical consensus is hard, especially in a project as large as Kubernetes, I believe doing so is critical for healthy communities.

 

Why not vote "-1"? The revised CNCF Sandbox project guidelines lower the bar for sandbox, making it a *tool* that can be used by anyone who needs a neutral place to host IP and collaborate on new projects with minimal overhead rather then a stepping stone towards incubation/graduation. This lets the TOC and CNCF SIGs to be more discriminating for incubation/graduation projects (only accept projects at that level that "makes sense in the CNCF ecosystem") while allowing the sandbox to serve as a test bed for innovation.

 


Matt Farina
 

I expect there are many people in this thread that lack some context. So, I figured I'd try to add some. If someone has more to add or a correction please do so. These are intermixed with some of my personal opinions, of course.

k3s feels very similar to Kind (https://github.com/kubernetes-sigs/kind) and minikube (https://github.com/kubernetes/minikube) both of which are Kubernetes sub-projects.

There are two things about this that jump out to me.

First, kind, k3s, and minikube are different in ways that are important to those who use them. Those subtle differences matter. This happens with other distros, too.

kind was developed with Kubernetes test infrastructure automation in mind. In fact, the docs say "kind was primarily designed for testing Kubernetes itself, but may be used for local development or CI.". Minikube, which has been around a lot longer, targets Kubernetes application developers. Their docs note, "minikube's primary goals are to be the best tool for local Kubernetes application development and to support all Kubernetes features that fit."

This distinction is important as we see people performing different roles. Tools for those different roles will likely look different in the end.

k3s targets a different group from these other two. Their docs note, "The certified Kubernetes distribution built for IoT & Edge computing". This is a different situation from the other two and I would expect the experiencing of using it to look different.

It may feel the same on the surface but the subtle goals are going to lead it in some different directions. Those are good as one size does not fit all.

Second, while minikube and kind are k8s sub-projects that doesn't mean all distros that fall under the CNCF need to be. kind you would expect to be be part of the kubernetes project because it was developed with testing kubernetes itself in mind. If minikube were started today it may have been something entirely separate from the kubernetes project. Over the years we have had discussions and debates on this theoretical topic.

The idea that all of these things must or should be part of the Kubernetes project doesn't fit with the discussions the k8s community has had over the years. In fact, people come to Kubernetes with ideas for changes and we regularly tell them to do them as part a different project rather than in k8s.

I'm surprised to see a push for distros that are open source to be part of the k8s project if they are to be in the CNCF.

I would like to to see k3s follow the same approach as those projects and apply to become a k8s sub-project first, rather than a standalone CNCF sandbox project.
This would ensure consistency, and give me confidence that Kubernetes experts have reviewed the project. Even if the k8s community ultimately says no, it wouldn't mean automatic no for a stand alone CNCF project, but would provide valuable insight for a CNCF application.

I would prefer to see diversity over consistency. If a distribution is conformant (we have tests for that) there should be room for diversity in the way things are done. Sometimes this will be experiments. Sometimes that will mean one distro is not consistent with another. For example, some distros ship with CRDs pre-installed which means an extended API. Sometimes that will mean someone swapped out a component with another version that uses the same APIs.

The sandbox is a great please to experiment in a cross company way.

The k8s community is amazingly busy. The people are busy. I wouldn't want to put more on their plates but rather take it off. Enabling groups to operate autonomously from each other or in a loosely coupled manner helps with that. Sub-projects of Kubernetes are limited in their ability to do that. A product of Kubernetes trying to efficiently handle the scale of people and decisions.

Being a separate CNCF project enables a loose coupling. It doesn't add work to an overworked k8s community but enables collaboration. This is an often overlooked element.

Open source projects aren't companies that fall along nice divisional lines. They are far more organic while variance in people, process, and ideas flourish. A fertile place for that is important.

On Mon, Aug 3, 2020, at 5:06 AM, Saad Ali via lists.cncf.io wrote:
Abstain

Unfortunately I missed the "Joint CNCF TOC/Kubernetes Steering Committee" meeting. But I talked to some folks afterwards to fill me in.

Overall, I'd like to avoid setting precedent that a project unable to generate consensus within the Kubernetes community, can bypass the community by going straight to the CNCF.
While that may not be what happened here, k3s feels very similar to Kind (https://github.com/kubernetes-sigs/kind) and minikube (https://github.com/kubernetes/minikube) both of which are Kubernetes sub-projects.
I would like to to see k3s follow the same approach as those projects and apply to become a k8s sub-project first, rather than a standalone CNCF sandbox project.
This would ensure consistency, and give me confidence that Kubernetes experts have reviewed the project. Even if the k8s community ultimately says no, it wouldn't mean automatic no for a stand alone CNCF project, but would provide valuable insight for a CNCF application.
While I realize that building technical consensus is hard, especially in a project as large as Kubernetes, I believe doing so is critical for healthy communities.

Why not vote "-1"? The revised CNCF Sandbox project guidelines lower the bar for sandbox, making it a *tool* that can be used by anyone who needs a neutral place to host IP and collaborate on new projects with minimal overhead rather then a stepping stone towards incubation/graduation. This lets the TOC and CNCF SIGs to be more discriminating for incubation/graduation projects (only accept projects at that level that "makes sense in the CNCF ecosystem") while allowing the sandbox to serve as a test bed for innovation.


es Zou
 

I don't think it needs to be accepted as a sandbox project. 
1. What is the purpose of acceptance? 
2. What is the difference between it and k8s? 
Finally, sandbox should not be abused.

-1

정철 <ivy@...> 于2020年8月8日周六 上午10:59写道:


+1 non-binding

Thank you,
Acornsoft,
Alex

2020년 8월 7일 (금) 오후 11:02, Jimmy Song <jimmysong@...>님이 작성:
+1 non-binding

On Aug 7, 2020, at 1:48 PM, daxmc99 <daxmc99@...> wrote:

+1 NB


정철
 


+1 non-binding

Thank you,
Acornsoft,
Alex

2020년 8월 7일 (금) 오후 11:02, Jimmy Song <jimmysong@...>님이 작성:

+1 non-binding

On Aug 7, 2020, at 1:48 PM, daxmc99 <daxmc99@...> wrote:

+1 NB


Tina Tsou
 

Dear all,

 

+1, NB.

 

 

Thank you,

Tina Tsou

Enterprise Architect

Arm

tina.tsou@...

+1 (408)931-3833

 

From: cncf-toc@... <cncf-toc@...> On Behalf Of St Leger, Jim via lists.cncf.io
Sent: Friday, August 7, 2020 3:59 PM
To: Amye Scavarda Perrin <ascavarda@...>; CNCF TOC <cncf-toc@...>
Subject: Re: [cncf-toc] [VOTE] k3s for Sandbox

 

+1 non-binding.

I've been on a bit of a pseudo-vacation. Not sure votes are all tallied and done (e.g. status of this vote) or still being gathered. My positive sentiment nonetheless.

Best,
Jim

 

 

From: cncf-toc@... <cncf-toc@...> On Behalf Of Amye Scavarda Perrin
Sent: Friday, July 31, 2020 8:05 AM
To: CNCF TOC <cncf-toc@...>
Subject: [cncf-toc] [VOTE] k3s for Sandbox

 

k3s has applied for inclusion into the sandbox: https://github.com/cncf/toc/pull/447.

Liz Rice has called for the vote: https://lists.cncf.io/g/cncf-toc/message/5081

Please vote (+1/0/-1) by replying to this thread.

Remember that the TOC has binding votes only, but we do appreciate non-binding votes from the community as a sign of support!

 

--

Amye Scavarda Perrin | Program Manager | amye@...

IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.


St Leger, Jim
 

+1 non-binding.

I've been on a bit of a pseudo-vacation. Not sure votes are all tallied and done (e.g. status of this vote) or still being gathered. My positive sentiment nonetheless.

Best,
Jim

 

 

From: cncf-toc@... <cncf-toc@...> On Behalf Of Amye Scavarda Perrin
Sent: Friday, July 31, 2020 8:05 AM
To: CNCF TOC <cncf-toc@...>
Subject: [cncf-toc] [VOTE] k3s for Sandbox

 

k3s has applied for inclusion into the sandbox: https://github.com/cncf/toc/pull/447.

Liz Rice has called for the vote: https://lists.cncf.io/g/cncf-toc/message/5081

Please vote (+1/0/-1) by replying to this thread.

Remember that the TOC has binding votes only, but we do appreciate non-binding votes from the community as a sign of support!

 

--

Amye Scavarda Perrin | Program Manager | amye@...


Jimmy Song <jimmysong@...>
 

+1 non-binding

On Aug 7, 2020, at 1:48 PM, daxmc99 <daxmc99@...> wrote:

+1 NB


Dax McDonald
 

+1 NB


Gadi Naor
 

-1 NB ... from a security perspective, i cant up vote this project despite my appreciation - at this point I am concerned  on how a fork maintain security patches of mainline k8s.

On Tue, Aug 4, 2020 at 12:03 AM Jimmy Song <jimmysong@...> wrote:
+1 non-binding

On Aug 2, 2020, at 10:42 AM, wuheng <wuheng@...> wrote:

+1 NB

发自 WPS邮箱客戶端
在 Mofei Zhang <mofei2816@...>,2020年8月1日 19:34写道:

+1 NB


On Jul 31, 2020, at 23:04, Amye Scavarda Perrin <ascavarda@...> wrote:

k3s has applied for inclusion into the sandbox: https://github.com/cncf/toc/pull/447.

Liz Rice has called for the vote: https://lists.cncf.io/g/cncf-toc/message/5081

Please vote (+1/0/-1) by replying to this thread.

Remember that the TOC has binding votes only, but we do appreciate non-binding votes from the community as a sign of support!

--
Amye Scavarda Perrin | Program Manager | amye@...

Mofei Zhang
Architect , JD.com
Maintainer , ChubaoFS


--
Gadi NaorCTO

US.   2443 Fillmore St, San Francisco, CA, 94115
IL.    5 Miconis St, Tel Aviv, 6777214   
M. +972-52-6618811
Web.      www.alcide.io
GitHub. github.com/alcideio

Follow us on LinkedInFollow us on Twitter 


Securing Kubernetes & Service Mesh.
Anywhere.
Bridging Security & DevOps.







김 진용 <jykim@...>
 

+1 NB

 

 

JINYONG KIM 김진용

CEO | 대표이사

 

NexCloud   http://www.nexcloud.co.kr http://www.nexclipper.io

O 02-533-8622  M +82 10-2048-8697  F 02-533-8623
06192 서울특별시 강남구 선릉로 428

 

 

 

 

보낸 사람: "cncf-toc@..." <cncf-toc@...>(wuheng <wuheng@...> 대신)
날짜: 2020 8 2 일요일 오전 11:43
받는 사람: "mofei2816@..." <mofei2816@...>, "ascavarda@..." <ascavarda@...>
참조: CNCF TOC <cncf-toc@...>
주제: Re: [cncf-toc] [VOTE] k3s for Sandbox

 

+1 NB

 

WPS箱客戶端

Mofei Zhang <mofei2816@...>202081 19:34道:

+1 NB

 



On Jul 31, 2020, at 23:04, Amye Scavarda Perrin <ascavarda@...> wrote:

 

k3s has applied for inclusion into the sandbox: https://github.com/cncf/toc/pull/447.

Liz Rice has called for the vote: https://lists.cncf.io/g/cncf-toc/message/5081

Please vote (+1/0/-1) by replying to this thread.

Remember that the TOC has binding votes only, but we do appreciate non-binding votes from the community as a sign of support!

 

--

Amye Scavarda Perrin | Program Manager | amye@...

 

Mofei Zhang

Architect , JD.com

Maintainer , ChubaoFS

 


Jimmy Song <jimmysong@...>
 

+1 non-binding

On Aug 2, 2020, at 10:42 AM, wuheng <wuheng@...> wrote:

+1 NB

发自 WPS邮箱客戶端
在 Mofei Zhang <mofei2816@...>,2020年8月1日 19:34写道:

+1 NB


On Jul 31, 2020, at 23:04, Amye Scavarda Perrin <ascavarda@...> wrote:

k3s has applied for inclusion into the sandbox: https://github.com/cncf/toc/pull/447.

Liz Rice has called for the vote: https://lists.cncf.io/g/cncf-toc/message/5081

Please vote (+1/0/-1) by replying to this thread.

Remember that the TOC has binding votes only, but we do appreciate non-binding votes from the community as a sign of support!

--
Amye Scavarda Perrin | Program Manager | amye@...

Mofei Zhang
Architect , JD.com
Maintainer , ChubaoFS



Justin Cormack
 

+1 binding

k3s has made it clear that it is not a fork and that is not the road it is going down.
It is a distro, and I feel that distros are valuable work in open source, with many examples.
The sandbox is a place for experiments, and allowing it in does not prejudge what road it may
take from here on.

Justin


On Fri, Jul 31, 2020 at 4:04 PM Amye Scavarda Perrin <ascavarda@...> wrote:
k3s has applied for inclusion into the sandbox: https://github.com/cncf/toc/pull/447.

Liz Rice has called for the vote: https://lists.cncf.io/g/cncf-toc/message/5081

Please vote (+1/0/-1) by replying to this thread.

Remember that the TOC has binding votes only, but we do appreciate non-binding votes from the community as a sign of support!

--
Amye Scavarda Perrin | Program Manager | amye@...


Darren Shepherd
 

I'm probably jumping into the conversation uninformed and without context.  I just want to make it very clear k3s is in no way shape or form a fork.  There is nothing technically correct about calling it a fork. While k3s does include some modifications to k8s code (about 300 lines or less), it is completely inline with the spirit of Linux distributions that carry their own patches for upstream components.  The patches we carry largely have to do with enabling how k3s packages k8s (single binary).

Furthermore k3s has a proven track record already of keeping inline with every k8s minor and patch release. Patch releases are released withing days or in the case of a CVE the same day.  Minor releases sometimes take a couple weeks to deliver, but we are trying to get that to day. We have zero intention of diverging from upstream.

Please, please do not call k3s fork. It isn't a fork. I understand why some might have come to that conclusion, but it's neither technically true or the intention of the project.

Darren


From: cncf-toc@... <cncf-toc@...> on behalf of Bob Wise via lists.cncf.io <bob=bobsplanet.com@...>
Sent: Monday, August 3, 2020 6:51 AM
To: alexis richardson <alexis@...>
Cc: aprokharchyk@... <aprokharchyk@...>; Joe Beda <jbeda@...>; wisebob@... <wisebob@...>; Amye Scavarda Perrin <ascavarda@...>; CNCF TOC <cncf-toc@...>
Subject: Re: [cncf-toc] [VOTE] k3s for Sandbox
 
K3s is a distro, and it is a fork as well. The "opinionated" description always applies to distros (which can be more or less opinionated, of course). If the opinion included things like backported security patches (does it?) that are available upstream but not for all versions, then I'd be inclined to agree that the spirit of the distro is not to be a fork.

That does not appear to be the case for K3s, which is a conformant fork. The comments about encouraging mending of fences and contribution upstream are further evidence of the forked nature of the project.

The TOC has distinct if related questions to address:

1) Will the TOC sponsor project forks into sandbox?
2) Will the TOC sponsor project distros into sandbox?
3) Is there something nuanced about sponsoring these into sandbox vs a track towards incubation and graduation?  If the purpose of the sandboxing of K3s were to give it a more neutral home while the forks are upstreamed and fences are mended, then that seems like a reasonable use of sandbox. If the track here is to put a fork onto a graduation path then that is much more concerning.
4) Kubernetes conformance tests are what determine distro eligibility. These tests are always evolving. What happens if a sponsored distro (or fork) stops passing conformance?

FWIW, I agree on the k8s sandbox point.

-Bob


On Fri, Jul 31, 2020 at 11:38 AM alexis richardson <alexis@...> wrote:
Currently k3s is a distro of k8s.

We used to have a k8s sandbox.  If we still did then k3s would happily live there as a way to show k8s how to be a better project.


On Fri, Jul 31, 2020 at 7:17 PM Alena Prokharchyk via lists.cncf.io <aprokharchyk=apple.com@...> wrote:
k3s is not a fork of Kubernetes. It is an opinionated way of delivering Kubernetes to IoT and Edge devices. Talking to Kubernetes steering committee made it clear that Kubernetes main design principle is extensibility, and the core system will be maintained to support Kubernetes development/deployment in a generic and configurable way. Therefore projects like k3s could benefit the ecosystem by expanding Kubernetes adoption footprint while remaining standalone.

For the areas where k3s maintainers can contribute back to Kubernetes, we should strongly encourage them to do so. Mending the relationship with Kubernetes community should be the first priority for the health of the project. By accepting k3s to Sandbox we get an opportunity to advise on its contributor experience, sustainability and governance. 

-alena

On Jul 31, 2020, at 11:07 AM, alexis richardson <alexis@...> wrote:

-1, nb

+1 to joe & bob comments.
I am very keen to see k3s initiative do well
I am even more keen to see it feed back into K8s

let's find a way to make this work!  I don't know the answer and recognise how unfair this probably seems to the k3s folks





On Fri, Jul 31, 2020 at 7:04 PM Joe Beda <jbeda@...> wrote:
-1 non-binding

 

My concerns echo Bob’s.

 

There is a ton to like about k3s.  There is a super enthusiastic user community and it has pioneered bringing together an opinionated set of components with a streamlined experience.  That is awesome!

 

Concrete concerns:
  1. Is k3s a distribution?  Many people publicly refer to it as such. The project page itself (k3s.io) has a headline saying “The certified Kubernetes distribution built for IoT & Edge computing”.
  2. Should the CNCF be a place to host distributions? (purpose discussion for the TOC)
  3. The name is very confusing.  Kubernetes and k8s are synonymous.  In fact, “k8s” is a registered trademark of the LF (https://www.linuxfoundation.org/trademark-list/).  I’m not a lawyer, but these clearly commercially overlap and there is confusion in the marketplace. The LF may need to take action here regardless of the TOC decision.
  4. The places where k3s has made progress has, traditionally, included essentially forking parts of k8s and rebuilding.  That forking has gotten thinner over time but is still there.  There are promises around pushing changes upstream, but, to my knowledge, that has been minimal.  The relationship there is fraught with a lot of history of friction and conflict.
  5. [point in time concern] The project is still pretty entangled with the rest of Rancher.  This can be solved but obviously hasn’t been a priority.  An example is that the documentation for k3s is part of the Rancher docs repo.  Would the docs be included in the assets that come into the CNCF?
    1. The repo is also part of the Rancher org.  The set of code owners is hidden and looks to be driven exclusively by Rancher (https://github.com/rancher/k3s/blob/master/CODEOWNERS).

 

There are a lot of thorny issues here. I have confidence in the TOC to be able to detangle these.

 

Thanks,
Joe

 

From: cncf-toc@... <cncf-toc@...>
Date: Friday, July 31, 2020 at 9:14 AM
To: aprokharchyk@... <aprokharchyk@...>, Amye Scavarda Perrin <ascavarda@...>
Cc: CNCF TOC <cncf-toc@...>
Subject: [Suspected Spam] Re: [cncf-toc] [VOTE] k3s for Sandbox
-1 non-binding.

 

I’m deeply concerned about the idea that CNCF is accepting what appears to be a Kubernetes fork into sandbox.

 

The statements about “encouraging upstream” seem like good intentions only.

 

Kubernetes as a project has, in my opinion, succeeded in part because of the community dedication to staying upstream and not forking.

 

For clarity, I would be strongly in favor of k3s being part of Kubernetes upstream.

 

-Bob

 

 

 

 

From: <cncf-toc@...> on behalf of "Alena Prokharchyk via lists.cncf.io" <aprokharchyk=apple.com@...>
Reply-To: "aprokharchyk@..." <aprokharchyk@...>
Date: Friday, July 31, 2020 at 9:09 AM
To: Amye Scavarda Perrin <ascavarda@...>
Cc: CNCF TOC <cncf-toc@...>
Subject: RE: [EXTERNAL] [cncf-toc] [VOTE] k3s for Sandbox

 

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.

 

+1 binding.

 

-alena.



On Jul 31, 2020, at 8:04 AM, Amye Scavarda Perrin <ascavarda@...> wrote:

 

k3s has applied for inclusion into the sandbox: https://github.com/cncf/toc/pull/447.

Liz Rice has called for the vote: https://lists.cncf.io/g/cncf-toc/message/5081 

Please vote (+1/0/-1) by replying to this thread.

Remember that the TOC has binding votes only, but we do appreciate non-binding votes from the community as a sign of support!

 

--
Amye Scavarda Perrin | Program Manager | amye@...

 

 





Brendan Burns
 

+1, binding.



From: cncf-toc@... <cncf-toc@...> on behalf of Sheng Liang via lists.cncf.io <sheng=rancher.com@...>
Sent: Sunday, August 2, 2020 7:47 AM
To: liz@... <liz@...>
Cc: Amye Scavarda Perrin <ascavarda@...>; CNCF TOC <cncf-toc@...>
Subject: [EXTERNAL] Re: [cncf-toc] [VOTE] k3s for Sandbox
 
+1 binding

On Aug 2, 2020 12:13 AM, "Liz Rice via lists.cncf.io" <liz=lizrice.com@...> wrote:
+1 binding



On Sat, 1 Aug 2020 at 09:22, Richard Hartmann <richih@...> wrote:
+1 NB

I see it as a learning tool, which has merit in and as of itself.
Similar to how Cortex and Thanos help shape the future for Prometheus,
k3s can act as a canonical proving ground of sorts before taking
things upstream.

While not distinct projects themselves, both prometheus-community and
prometheus-operator (recently moved out of its old coreos/ home) also
help shape and advance core Prometheus. k3s just happens to have more
people on it than the two prometheus-* mentioned here.

On Fri, Jul 31, 2020 at 5:04 PM Amye Scavarda Perrin
<ascavarda@...> wrote:
>
> k3s has applied for inclusion into the sandbox: https://github.com/cncf/toc/pull/447.
>
> Liz Rice has called for the vote: https://lists.cncf.io/g/cncf-toc/message/5081
>
> Please vote (+1/0/-1) by replying to this thread.
>
> Remember that the TOC has binding votes only, but we do appreciate non-binding votes from the community as a sign of support!
>
> --
> Amye Scavarda Perrin | Program Manager | amye@...
>




Liz Rice
 

Per Bob's point 3, k3s want a neutral home for further experimentation and collaboration; in my opinion it is undoubtedly cloud native, and it solves a particular problem in a way that lots of folks find useful, so (again IMO) CNCF is the natural organization for it to get that neutral home. 

Should that be within k8s or as a standalone project? After the conversation with k8s Steering I believe a separate project is the path where k3s is most likely to succeed at this stage, and it doesn't place any new requirements or burden on the k8s team. As Alena, Michelle and others have mentioned, being part of the same foundation gives us an opportunity for better communication and alignment between projects going forward. 

k3s have stated their intention to remain conformant, and that's a big part of why I think there is potential for success here. 


On Mon, Aug 3, 2020 at 2:51 PM Bob Wise <bob@...> wrote:
K3s is a distro, and it is a fork as well. The "opinionated" description always applies to distros (which can be more or less opinionated, of course). If the opinion included things like backported security patches (does it?) that are available upstream but not for all versions, then I'd be inclined to agree that the spirit of the distro is not to be a fork.

That does not appear to be the case for K3s, which is a conformant fork. The comments about encouraging mending of fences and contribution upstream are further evidence of the forked nature of the project.

The TOC has distinct if related questions to address:

1) Will the TOC sponsor project forks into sandbox?
2) Will the TOC sponsor project distros into sandbox?
3) Is there something nuanced about sponsoring these into sandbox vs a track towards incubation and graduation?  If the purpose of the sandboxing of K3s were to give it a more neutral home while the forks are upstreamed and fences are mended, then that seems like a reasonable use of sandbox. If the track here is to put a fork onto a graduation path then that is much more concerning.
4) Kubernetes conformance tests are what determine distro eligibility. These tests are always evolving. What happens if a sponsored distro (or fork) stops passing conformance?

FWIW, I agree on the k8s sandbox point.

-Bob


On Fri, Jul 31, 2020 at 11:38 AM alexis richardson <alexis@...> wrote:
Currently k3s is a distro of k8s.

We used to have a k8s sandbox.  If we still did then k3s would happily live there as a way to show k8s how to be a better project.


On Fri, Jul 31, 2020 at 7:17 PM Alena Prokharchyk via lists.cncf.io <aprokharchyk=apple.com@...> wrote:
k3s is not a fork of Kubernetes. It is an opinionated way of delivering Kubernetes to IoT and Edge devices. Talking to Kubernetes steering committee made it clear that Kubernetes main design principle is extensibility, and the core system will be maintained to support Kubernetes development/deployment in a generic and configurable way. Therefore projects like k3s could benefit the ecosystem by expanding Kubernetes adoption footprint while remaining standalone.

For the areas where k3s maintainers can contribute back to Kubernetes, we should strongly encourage them to do so. Mending the relationship with Kubernetes community should be the first priority for the health of the project. By accepting k3s to Sandbox we get an opportunity to advise on its contributor experience, sustainability and governance. 

-alena

On Jul 31, 2020, at 11:07 AM, alexis richardson <alexis@...> wrote:

-1, nb

+1 to joe & bob comments.
I am very keen to see k3s initiative do well
I am even more keen to see it feed back into K8s

let's find a way to make this work!  I don't know the answer and recognise how unfair this probably seems to the k3s folks





On Fri, Jul 31, 2020 at 7:04 PM Joe Beda <jbeda@...> wrote:
-1 non-binding

 

My concerns echo Bob’s.

 

There is a ton to like about k3s.  There is a super enthusiastic user community and it has pioneered bringing together an opinionated set of components with a streamlined experience.  That is awesome!

 

Concrete concerns:
  1. Is k3s a distribution?  Many people publicly refer to it as such. The project page itself (k3s.io) has a headline saying “The certified Kubernetes distribution built for IoT & Edge computing”.
  2. Should the CNCF be a place to host distributions? (purpose discussion for the TOC)
  3. The name is very confusing.  Kubernetes and k8s are synonymous.  In fact, “k8s” is a registered trademark of the LF (https://www.linuxfoundation.org/trademark-list/).  I’m not a lawyer, but these clearly commercially overlap and there is confusion in the marketplace. The LF may need to take action here regardless of the TOC decision.
  4. The places where k3s has made progress has, traditionally, included essentially forking parts of k8s and rebuilding.  That forking has gotten thinner over time but is still there.  There are promises around pushing changes upstream, but, to my knowledge, that has been minimal.  The relationship there is fraught with a lot of history of friction and conflict.
  5. [point in time concern] The project is still pretty entangled with the rest of Rancher.  This can be solved but obviously hasn’t been a priority.  An example is that the documentation for k3s is part of the Rancher docs repo.  Would the docs be included in the assets that come into the CNCF?
    1. The repo is also part of the Rancher org.  The set of code owners is hidden and looks to be driven exclusively by Rancher (https://github.com/rancher/k3s/blob/master/CODEOWNERS).

 

There are a lot of thorny issues here. I have confidence in the TOC to be able to detangle these.

 

Thanks,
Joe

 

From: cncf-toc@... <cncf-toc@...>
Date: Friday, July 31, 2020 at 9:14 AM
To: aprokharchyk@... <aprokharchyk@...>, Amye Scavarda Perrin <ascavarda@...>
Cc: CNCF TOC <cncf-toc@...>
Subject: [Suspected Spam] Re: [cncf-toc] [VOTE] k3s for Sandbox
-1 non-binding.

 

I’m deeply concerned about the idea that CNCF is accepting what appears to be a Kubernetes fork into sandbox.

 

The statements about “encouraging upstream” seem like good intentions only.

 

Kubernetes as a project has, in my opinion, succeeded in part because of the community dedication to staying upstream and not forking.

 

For clarity, I would be strongly in favor of k3s being part of Kubernetes upstream.

 

-Bob

 

 

 

 

From: <cncf-toc@...> on behalf of "Alena Prokharchyk via lists.cncf.io" <aprokharchyk=apple.com@...>
Reply-To: "aprokharchyk@..." <aprokharchyk@...>
Date: Friday, July 31, 2020 at 9:09 AM
To: Amye Scavarda Perrin <ascavarda@...>
Cc: CNCF TOC <cncf-toc@...>
Subject: RE: [EXTERNAL] [cncf-toc] [VOTE] k3s for Sandbox

 

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.

 

+1 binding.

 

-alena.



On Jul 31, 2020, at 8:04 AM, Amye Scavarda Perrin <ascavarda@...> wrote:

 

k3s has applied for inclusion into the sandbox: https://github.com/cncf/toc/pull/447.

Liz Rice has called for the vote: https://lists.cncf.io/g/cncf-toc/message/5081 

Please vote (+1/0/-1) by replying to this thread.

Remember that the TOC has binding votes only, but we do appreciate non-binding votes from the community as a sign of support!

 

--
Amye Scavarda Perrin | Program Manager | amye@...

 

 





Bob Wise
 

K3s is a distro, and it is a fork as well. The "opinionated" description always applies to distros (which can be more or less opinionated, of course). If the opinion included things like backported security patches (does it?) that are available upstream but not for all versions, then I'd be inclined to agree that the spirit of the distro is not to be a fork.

That does not appear to be the case for K3s, which is a conformant fork. The comments about encouraging mending of fences and contribution upstream are further evidence of the forked nature of the project.

The TOC has distinct if related questions to address:

1) Will the TOC sponsor project forks into sandbox?
2) Will the TOC sponsor project distros into sandbox?
3) Is there something nuanced about sponsoring these into sandbox vs a track towards incubation and graduation?  If the purpose of the sandboxing of K3s were to give it a more neutral home while the forks are upstreamed and fences are mended, then that seems like a reasonable use of sandbox. If the track here is to put a fork onto a graduation path then that is much more concerning.
4) Kubernetes conformance tests are what determine distro eligibility. These tests are always evolving. What happens if a sponsored distro (or fork) stops passing conformance?

FWIW, I agree on the k8s sandbox point.

-Bob


On Fri, Jul 31, 2020 at 11:38 AM alexis richardson <alexis@...> wrote:
Currently k3s is a distro of k8s.

We used to have a k8s sandbox.  If we still did then k3s would happily live there as a way to show k8s how to be a better project.


On Fri, Jul 31, 2020 at 7:17 PM Alena Prokharchyk via lists.cncf.io <aprokharchyk=apple.com@...> wrote:
k3s is not a fork of Kubernetes. It is an opinionated way of delivering Kubernetes to IoT and Edge devices. Talking to Kubernetes steering committee made it clear that Kubernetes main design principle is extensibility, and the core system will be maintained to support Kubernetes development/deployment in a generic and configurable way. Therefore projects like k3s could benefit the ecosystem by expanding Kubernetes adoption footprint while remaining standalone.

For the areas where k3s maintainers can contribute back to Kubernetes, we should strongly encourage them to do so. Mending the relationship with Kubernetes community should be the first priority for the health of the project. By accepting k3s to Sandbox we get an opportunity to advise on its contributor experience, sustainability and governance. 

-alena

On Jul 31, 2020, at 11:07 AM, alexis richardson <alexis@...> wrote:

-1, nb

+1 to joe & bob comments.
I am very keen to see k3s initiative do well
I am even more keen to see it feed back into K8s

let's find a way to make this work!  I don't know the answer and recognise how unfair this probably seems to the k3s folks





On Fri, Jul 31, 2020 at 7:04 PM Joe Beda <jbeda@...> wrote:
-1 non-binding

 

My concerns echo Bob’s.

 

There is a ton to like about k3s.  There is a super enthusiastic user community and it has pioneered bringing together an opinionated set of components with a streamlined experience.  That is awesome!

 

Concrete concerns:
  1. Is k3s a distribution?  Many people publicly refer to it as such. The project page itself (k3s.io) has a headline saying “The certified Kubernetes distribution built for IoT & Edge computing”.
  2. Should the CNCF be a place to host distributions? (purpose discussion for the TOC)
  3. The name is very confusing.  Kubernetes and k8s are synonymous.  In fact, “k8s” is a registered trademark of the LF (https://www.linuxfoundation.org/trademark-list/).  I’m not a lawyer, but these clearly commercially overlap and there is confusion in the marketplace. The LF may need to take action here regardless of the TOC decision.
  4. The places where k3s has made progress has, traditionally, included essentially forking parts of k8s and rebuilding.  That forking has gotten thinner over time but is still there.  There are promises around pushing changes upstream, but, to my knowledge, that has been minimal.  The relationship there is fraught with a lot of history of friction and conflict.
  5. [point in time concern] The project is still pretty entangled with the rest of Rancher.  This can be solved but obviously hasn’t been a priority.  An example is that the documentation for k3s is part of the Rancher docs repo.  Would the docs be included in the assets that come into the CNCF?
    1. The repo is also part of the Rancher org.  The set of code owners is hidden and looks to be driven exclusively by Rancher (https://github.com/rancher/k3s/blob/master/CODEOWNERS).

 

There are a lot of thorny issues here. I have confidence in the TOC to be able to detangle these.

 

Thanks,
Joe

 

From: cncf-toc@... <cncf-toc@...>
Date: Friday, July 31, 2020 at 9:14 AM
To: aprokharchyk@... <aprokharchyk@...>, Amye Scavarda Perrin <ascavarda@...>
Cc: CNCF TOC <cncf-toc@...>
Subject: [Suspected Spam] Re: [cncf-toc] [VOTE] k3s for Sandbox
-1 non-binding.

 

I’m deeply concerned about the idea that CNCF is accepting what appears to be a Kubernetes fork into sandbox.

 

The statements about “encouraging upstream” seem like good intentions only.

 

Kubernetes as a project has, in my opinion, succeeded in part because of the community dedication to staying upstream and not forking.

 

For clarity, I would be strongly in favor of k3s being part of Kubernetes upstream.

 

-Bob

 

 

 

 

From: <cncf-toc@...> on behalf of "Alena Prokharchyk via lists.cncf.io" <aprokharchyk=apple.com@...>
Reply-To: "aprokharchyk@..." <aprokharchyk@...>
Date: Friday, July 31, 2020 at 9:09 AM
To: Amye Scavarda Perrin <ascavarda@...>
Cc: CNCF TOC <cncf-toc@...>
Subject: RE: [EXTERNAL] [cncf-toc] [VOTE] k3s for Sandbox

 

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.

 

+1 binding.

 

-alena.



On Jul 31, 2020, at 8:04 AM, Amye Scavarda Perrin <ascavarda@...> wrote:

 

k3s has applied for inclusion into the sandbox: https://github.com/cncf/toc/pull/447.

Liz Rice has called for the vote: https://lists.cncf.io/g/cncf-toc/message/5081 

Please vote (+1/0/-1) by replying to this thread.

Remember that the TOC has binding votes only, but we do appreciate non-binding votes from the community as a sign of support!

 

--
Amye Scavarda Perrin | Program Manager | amye@...