1. cncf-toc@lists.cncf.io
  2. Messages
   Date   
Re: Thoughts on KubeCon

Quinton Hoole
 

Thanks for the insightful and thought-provoking blog post Bryan.  I missed the call yesterday, but co-incidentally had been noodling with similar thoughts recently, as, anecdotally, I’m also not convinced that we have the best submission review outcomes today.  I think that introducing the the partially double-blind review process would be a great step forward, and may well obviate the need for complicated per-vendor limits.

I also think that it would be super-useful for submission rejection notices to be accompanied by a few brief reviewer notes (e.g. “too much marketing pitch”, “not open source”, “previously presented”, “duplicated submission”, “off topic" etc) to help submitters to improve their chances in future (and perhaps also clarify any possible misperceptions by reviewers, as the submissions are by necessity brief). As just one illustrative data point, all 10 of my submissions to KubeCon China and US were rejected, and none of the rejections seem explainable by any of the “how to improve the odds” guidelines.  So I have no idea what to do differently in future.

Q

From: <cncf-toc@...> on behalf of Bryan Cantrill <bryan@...>
Date: Wednesday, October 3, 2018 at 12:58
To: "anthony@..." <anthony@...>
Cc: CNCF TOC <cncf-toc@...>
Subject: Re: [cncf-toc] Thoughts on KubeCon


One per vendor might be too acute, as some vendors are doing much more than others.  But having some system that limits the number of submissions per vendor (and therefore force the vendors to adopt some process to determine their best submissions) would probably help -- and would also help address the too-low acceptance rate...

        - Bryan
 

On Wed, Oct 3, 2018 at 11:54 AM Anthony Skipper <anthony@...> wrote:
I would agree with double blind.  But a max of 1 talk per vendor might also go a long way. 

On Wed, Oct 3, 2018 at 2:47 PM Bryan Cantrill <bryan@...> wrote:

On the call yesterday, Dan asked me to send out my thoughts on double-blind reviewing.  My e-mail quickly turned into a blog entry:


Something that I probably didn't highlight well enough in there is Kathryn McKinley's excellent piece on double-blind review:


There are certainly lots of ways to attack this problem, but I view double-blind as an essential piece -- but probably not sufficient on its own.

         - Bryan
Re: Thoughts on KubeCon

William Morgan
 

I was a reviewer for both China and North America this year, and a double-blind academia reviewer earlier in life. I sent some feedback to the program committee which mostly echoed Brian's blog post:

Primarily, I think that there should be two distinct decisions: 1) is this going to be a "good" presentation, in isolation; and 2) is this presentation going to be good in the context of everything else that's happening at the conference. I think that third-party reviewers such as myself should only try to evaluate #1, and in a second step the conference organizers should evaluate #2. Then in #1 we could even make this a blind review process where the presenter name and company is hidden at review time, and in #2 you could be explicitly focused on making a good cohesive diverse conference. Right now those two competing desires are mixed together.

I also had some suggestions about how to make the scoring more deterministic across reviewers by providing a more explicit rubric.

Personally I'd like to see a much stronger emphasis on practitioner talks. "I used X in prod and here are the challenges we had to overcome." But IMO the most important first step is to have reliable and consistent reviewer scores, for which double blind reviews would really help.

-William


On Wed, Oct 3, 2018 at 11:47 AM Bryan Cantrill <bryan@...> wrote:

On the call yesterday, Dan asked me to send out my thoughts on double-blind reviewing.  My e-mail quickly turned into a blog entry:


Something that I probably didn't highlight well enough in there is Kathryn McKinley's excellent piece on double-blind review:


There are certainly lots of ways to attack this problem, but I view double-blind as an essential piece -- but probably not sufficient on its own.

         - Bryan
Re: Thoughts on KubeCon

Shannon Williams
 

+1

Best Regards,

Shannon Williams
+1 (650) 521-6902


On Oct 3, 2018, at 11:54 AM, Anthony Skipper <anthony@...> wrote:

I would agree with double blind.  But a max of 1 talk per vendor might also go a long way. 

On Wed, Oct 3, 2018 at 2:47 PM Bryan Cantrill <bryan@...> wrote:

On the call yesterday, Dan asked me to send out my thoughts on double-blind reviewing.  My e-mail quickly turned into a blog entry:


Something that I probably didn't highlight well enough in there is Kathryn McKinley's excellent piece on double-blind review:


There are certainly lots of ways to attack this problem, but I view double-blind as an essential piece -- but probably not sufficient on its own.

         - Bryan
Re: Thoughts on KubeCon

Bryan Cantrill <bryan@...>
 


One per vendor might be too acute, as some vendors are doing much more than others.  But having some system that limits the number of submissions per vendor (and therefore force the vendors to adopt some process to determine their best submissions) would probably help -- and would also help address the too-low acceptance rate...

        - Bryan
 

On Wed, Oct 3, 2018 at 11:54 AM Anthony Skipper <anthony@...> wrote:
I would agree with double blind.  But a max of 1 talk per vendor might also go a long way. 

On Wed, Oct 3, 2018 at 2:47 PM Bryan Cantrill <bryan@...> wrote:

On the call yesterday, Dan asked me to send out my thoughts on double-blind reviewing.  My e-mail quickly turned into a blog entry:


Something that I probably didn't highlight well enough in there is Kathryn McKinley's excellent piece on double-blind review:


There are certainly lots of ways to attack this problem, but I view double-blind as an essential piece -- but probably not sufficient on its own.

         - Bryan
Re: Thoughts on KubeCon

Dan Kohn <dan@...>
 

On Wed, Oct 3, 2018 at 3:20 PM Matt Farina <matt@...> wrote:
If we talk about vendor limits should we exclude SIG specific and project specific sessions? That is the intros and deep dives. That gives some orgs high numbers (like 18 of the vendor who has 41).

This has useful context for how talks are selected: https://www.cncf.io/blog/2018/05/29/get-your-kubecon-talk-accepted/

At a high level, the Intro/Deep Dive tracks are separate from the CFP tracks, and we calculate statistics separately.
 
Do existing sponsorship levels include numbers of speaking slots?

We have 6 diamond sponsors that each get a 5-minute sponsored keynote. All other keynotes and CFP tracks are rated by the program committee and then selected into tracks by the co-chairs.
--
Dan Kohn <dan@...>
Executive Director, Cloud Native Computing Foundation https://www.cncf.io
+1-415-233-1000 https://www.dankohn.com
Re: Thoughts on KubeCon

Matt Farina
 

If we talk about vendor limits should we exclude SIG specific and project specific sessions? That is the intros and deep dives. That gives some orgs high numbers (like 18 of the vendor who has 41).

Do existing sponsorship levels include numbers of speaking slots?

-- 
Matt Farina
mattfarina.com
Re: Thoughts on KubeCon

Nick Chase
 

Even a max of 3-5 from one vendor would be a significant difference from the 68 from one company, 41 from another....

----  Nick


On 10/3/2018 2:54 PM, Anthony Skipper wrote:
I would agree with double blind.  But a max of 1 talk per vendor might also go a long way. 

On Wed, Oct 3, 2018 at 2:47 PM Bryan Cantrill <bryan@...> wrote:

On the call yesterday, Dan asked me to send out my thoughts on double-blind reviewing.  My e-mail quickly turned into a blog entry:


Something that I probably didn't highlight well enough in there is Kathryn McKinley's excellent piece on double-blind review:


There are certainly lots of ways to attack this problem, but I view double-blind as an essential piece -- but probably not sufficient on its own.

         - Bryan

--
Nick Chase, Head of Technical and Marketing Content, Mirantis
Editor in Chief, Open Cloud Digest Author, Machine Learning for Mere Mortals
Re: Thoughts on KubeCon

Anthony Skipper <anthony@...>
 

I would agree with double blind.  But a max of 1 talk per vendor might also go a long way. 


On Wed, Oct 3, 2018 at 2:47 PM Bryan Cantrill <bryan@...> wrote:

On the call yesterday, Dan asked me to send out my thoughts on double-blind reviewing.  My e-mail quickly turned into a blog entry:


Something that I probably didn't highlight well enough in there is Kathryn McKinley's excellent piece on double-blind review:


There are certainly lots of ways to attack this problem, but I view double-blind as an essential piece -- but probably not sufficient on its own.

         - Bryan
Thoughts on KubeCon

Bryan Cantrill <bryan@...>
 


On the call yesterday, Dan asked me to send out my thoughts on double-blind reviewing.  My e-mail quickly turned into a blog entry:


Something that I probably didn't highlight well enough in there is Kathryn McKinley's excellent piece on double-blind review:


There are certainly lots of ways to attack this problem, but I view double-blind as an essential piece -- but probably not sufficient on its own.

         - Bryan
Re: Moving to written proposals for projects over presentations

alexis richardson
 

thanks Michael, good write up.

I really like the overall thinking here.  We are not raising the bar for adoption, but we are asking for clarity of thought and planning for early projects.  That will make it much easier for the community & foundation to help them (if we can...) and to measure success.


On Tue, Oct 2, 2018 at 7:36 PM Michael Ducy <michael.ducy@...> wrote:
To add a bit of color on the Falco Sandbox proposal and presentation. 

I started with the proposal document first because I felt that the problem we were trying to solve may not necessarily be well understood if we only did a presentation. I also wanted to have any of the questions/requirements for a Sandbox project answered ahead of the presentation. That way any TOC members could be confident that we met the requirements and we weren't wasting anyone's time. If questions or confusion came up, I had the proposal document to refer people back to. Lastly, I wanted people to clearly understand where we were fitting in the Cloud Native ecosystem, and the value we were providing. I felt like that would be harder to get across in the presentation. Personally, I felt that these were the barriers that we needed to overcome to get the TOC sponsors required. 

From the proposal I built the presentation, which felt like it naturally came out of the proposal. Each section became a slide (or two), and we had a much more clear story to tell on the slides as it was right there in the document as well. The proposal document also gave us a much more clear story to tell when we presented to the TOC.  

I'm not sure if most projects present first then write the proposal, but if not, it might be useful to flip it around. 

Michael



On Tue, Oct 2, 2018 at 12:10 PM Chris Aniszczyk <caniszczyk@...> wrote:
I agree requiring the writing up front (instead of after the presentation) can be useful, here are some good examples imho:

Falco (sandbox)

Rook (sandbox->incubation)

If we want a specific template happy to hear ideas, we can put it in the TOC repo.


On Tue, Oct 2, 2018 at 11:06 AM Camille Fournier <skamille@...> wrote:
Chris pointed out in chat that groups already are writing up docs for their proposals. However, the problem in my mind is that first we see a slide deck style presentation, then it is followed by a written doc.

I can't speak for Bob, but as someone who heavily adopted Amazon-style paper writing over presentations for much of my internal team, I far prefer reading thoroughly-written docs to watching slide shows. Right now, the details that we get in a lot of the proposals we vote on are not nearly as thorough as what we see in the presentations, and if we tried to replace the presentations with the proposal docs, we would not have much to go on. 

A typical design doc that I would review would have information like:
Summary
Project goals (possibly including user scenarios this is aimed at addressing)
Project non-goals
High-level design
Roadmap

There's no one exact way to write one, but if we want to move to a writing-driven review process at least for sandbox projects, we should agree on more questions we want answered as part of the doc and length recommendations (and possibly restrictions). I feel like this doc is going to be a hybrid product/tech design doc, so if there are any product managers who want to chime in with further suggestions I'm all ears.

C




--
Chris Aniszczyk (@cra) | +1-512-961-6719
Re: Moving to written proposals for projects over presentations

Michael Ducy
 

To add a bit of color on the Falco Sandbox proposal and presentation. 

I started with the proposal document first because I felt that the problem we were trying to solve may not necessarily be well understood if we only did a presentation. I also wanted to have any of the questions/requirements for a Sandbox project answered ahead of the presentation. That way any TOC members could be confident that we met the requirements and we weren't wasting anyone's time. If questions or confusion came up, I had the proposal document to refer people back to. Lastly, I wanted people to clearly understand where we were fitting in the Cloud Native ecosystem, and the value we were providing. I felt like that would be harder to get across in the presentation. Personally, I felt that these were the barriers that we needed to overcome to get the TOC sponsors required. 

From the proposal I built the presentation, which felt like it naturally came out of the proposal. Each section became a slide (or two), and we had a much more clear story to tell on the slides as it was right there in the document as well. The proposal document also gave us a much more clear story to tell when we presented to the TOC.  

I'm not sure if most projects present first then write the proposal, but if not, it might be useful to flip it around. 

Michael



On Tue, Oct 2, 2018 at 12:10 PM Chris Aniszczyk <caniszczyk@...> wrote:
I agree requiring the writing up front (instead of after the presentation) can be useful, here are some good examples imho:

Falco (sandbox)

Rook (sandbox->incubation)

If we want a specific template happy to hear ideas, we can put it in the TOC repo.


On Tue, Oct 2, 2018 at 11:06 AM Camille Fournier <skamille@...> wrote:
Chris pointed out in chat that groups already are writing up docs for their proposals. However, the problem in my mind is that first we see a slide deck style presentation, then it is followed by a written doc.

I can't speak for Bob, but as someone who heavily adopted Amazon-style paper writing over presentations for much of my internal team, I far prefer reading thoroughly-written docs to watching slide shows. Right now, the details that we get in a lot of the proposals we vote on are not nearly as thorough as what we see in the presentations, and if we tried to replace the presentations with the proposal docs, we would not have much to go on. 

A typical design doc that I would review would have information like:
Summary
Project goals (possibly including user scenarios this is aimed at addressing)
Project non-goals
High-level design
Roadmap

There's no one exact way to write one, but if we want to move to a writing-driven review process at least for sandbox projects, we should agree on more questions we want answered as part of the doc and length recommendations (and possibly restrictions). I feel like this doc is going to be a hybrid product/tech design doc, so if there are any product managers who want to chime in with further suggestions I'm all ears.

C




--
Chris Aniszczyk (@cra) | +1-512-961-6719
Re: Moving to written proposals for projects over presentations

Davanum Srinivas
 

Camille,

If we are looking for prior art to build templates on, the ASF incubator has a template [1], examples of proposals are at [2][3]. 

I like the sections in there on why the project wants to come to ASF, and the risks too.

Thanks,
Dims


On Tue, Oct 2, 2018 at 12:06 PM Camille Fournier <skamille@...> wrote:
Chris pointed out in chat that groups already are writing up docs for their proposals. However, the problem in my mind is that first we see a slide deck style presentation, then it is followed by a written doc.

I can't speak for Bob, but as someone who heavily adopted Amazon-style paper writing over presentations for much of my internal team, I far prefer reading thoroughly-written docs to watching slide shows. Right now, the details that we get in a lot of the proposals we vote on are not nearly as thorough as what we see in the presentations, and if we tried to replace the presentations with the proposal docs, we would not have much to go on. 

A typical design doc that I would review would have information like:
Summary
Project goals (possibly including user scenarios this is aimed at addressing)
Project non-goals
High-level design
Roadmap

There's no one exact way to write one, but if we want to move to a writing-driven review process at least for sandbox projects, we should agree on more questions we want answered as part of the doc and length recommendations (and possibly restrictions). I feel like this doc is going to be a hybrid product/tech design doc, so if there are any product managers who want to chime in with further suggestions I'm all ears.

C




--
Davanum Srinivas :: https://twitter.com/dims
Re: Moving to written proposals for projects over presentations

Chris Aniszczyk
 

I agree requiring the writing up front (instead of after the presentation) can be useful, here are some good examples imho:

Falco (sandbox)

Rook (sandbox->incubation)

If we want a specific template happy to hear ideas, we can put it in the TOC repo.


On Tue, Oct 2, 2018 at 11:06 AM Camille Fournier <skamille@...> wrote:
Chris pointed out in chat that groups already are writing up docs for their proposals. However, the problem in my mind is that first we see a slide deck style presentation, then it is followed by a written doc.

I can't speak for Bob, but as someone who heavily adopted Amazon-style paper writing over presentations for much of my internal team, I far prefer reading thoroughly-written docs to watching slide shows. Right now, the details that we get in a lot of the proposals we vote on are not nearly as thorough as what we see in the presentations, and if we tried to replace the presentations with the proposal docs, we would not have much to go on. 

A typical design doc that I would review would have information like:
Summary
Project goals (possibly including user scenarios this is aimed at addressing)
Project non-goals
High-level design
Roadmap

There's no one exact way to write one, but if we want to move to a writing-driven review process at least for sandbox projects, we should agree on more questions we want answered as part of the doc and length recommendations (and possibly restrictions). I feel like this doc is going to be a hybrid product/tech design doc, so if there are any product managers who want to chime in with further suggestions I'm all ears.

C




--
Chris Aniszczyk (@cra) | +1-512-961-6719
Moving to written proposals for projects over presentations

Camille Fournier
 

Chris pointed out in chat that groups already are writing up docs for their proposals. However, the problem in my mind is that first we see a slide deck style presentation, then it is followed by a written doc.

I can't speak for Bob, but as someone who heavily adopted Amazon-style paper writing over presentations for much of my internal team, I far prefer reading thoroughly-written docs to watching slide shows. Right now, the details that we get in a lot of the proposals we vote on are not nearly as thorough as what we see in the presentations, and if we tried to replace the presentations with the proposal docs, we would not have much to go on. 

A typical design doc that I would review would have information like:
Summary
Project goals (possibly including user scenarios this is aimed at addressing)
Project non-goals
High-level design
Roadmap

There's no one exact way to write one, but if we want to move to a writing-driven review process at least for sandbox projects, we should agree on more questions we want answered as part of the doc and length recommendations (and possibly restrictions). I feel like this doc is going to be a hybrid product/tech design doc, so if there are any product managers who want to chime in with further suggestions I'm all ears.

C
Re: TOC Agenda 10/2/2018

Quinton Hoole
 

I’ll be boarding a plane at 8AM, so I probably won’t be on the call today.

Q

Quinton Hoole

Technical Vice President

America Research Center

2330 Central Expressway, Santa Clara, CA 95050

Tel: 408-330-4721   Cell: 408-320-8917   Office # C2-27

Email: quinton.hoole@...   ID#Q00403160


From: <cncf-toc@...> on behalf of alexis richardson <alexis@...>
Date: Tuesday, October 2, 2018 at 02:43
To: Chris Aniszczyk <caniszczyk@...>
Cc: CNCF TOC <cncf-toc@...>
Subject: Re: [cncf-toc] TOC Agenda 10/2/2018

I have updated the Agenda


On Mon, 1 Oct 2018, 22:19 alexis richardson, <alexis@...> wrote:
Hi all

Apologies, there will be an updated Agenda tomorrow. Want to discuss some TOC business.  Will be asking project presentations to go on hold

Alexis

On Mon, 1 Oct 2018, 21:02 Chris Aniszczyk, <caniszczyk@...> wrote:
Here's the agenda deck for tomorrow: 


We will be hearing from the keycloak project on top of welcoming Cheryl to the CNCF who will be working on ways to improve our end user ecosystem.

See everyone tomorrow!


--
Chris Aniszczyk (@cra) | +1-512-961-6719
Re: TOC Agenda 10/2/2018

Richard Hartmann
 

I won't have time to listen in on the call, but I like what I am
seeing. Thanks Alexis and everyone else.

On Tue, Oct 2, 2018 at 11:43 AM alexis richardson <alexis@...> wrote:

I have updated the Agenda


On Mon, 1 Oct 2018, 22:19 alexis richardson, <alexis@...> wrote:

Hi all

Apologies, there will be an updated Agenda tomorrow. Want to discuss some TOC business. Will be asking project presentations to go on hold

Alexis

On Mon, 1 Oct 2018, 21:02 Chris Aniszczyk, <caniszczyk@...> wrote:

Here's the agenda deck for tomorrow:

https://docs.google.com/presentation/d/1Xt1xNSN8_pGuDLl5H8xEYToFss7VoIm7GBG0e_HrsLc/edit?ts=5bae170b#slide=id.g25ca91f87f_0_0

We will be hearing from the keycloak project on top of welcoming Cheryl to the CNCF who will be working on ways to improve our end user ecosystem.

See everyone tomorrow!


--
Chris Aniszczyk (@cra) | +1-512-961-6719
Re: TOC Agenda 10/2/2018

alexis richardson
 

I have updated the Agenda


On Mon, 1 Oct 2018, 22:19 alexis richardson, <alexis@...> wrote:
Hi all

Apologies, there will be an updated Agenda tomorrow. Want to discuss some TOC business.  Will be asking project presentations to go on hold

Alexis

On Mon, 1 Oct 2018, 21:02 Chris Aniszczyk, <caniszczyk@...> wrote:
Here's the agenda deck for tomorrow: 


We will be hearing from the keycloak project on top of welcoming Cheryl to the CNCF who will be working on ways to improve our end user ecosystem.

See everyone tomorrow!


--
Chris Aniszczyk (@cra) | +1-512-961-6719
Re: TOC Agenda 10/2/2018

alexis richardson
 

Hi all

Apologies, there will be an updated Agenda tomorrow. Want to discuss some TOC business.  Will be asking project presentations to go on hold

Alexis

On Mon, 1 Oct 2018, 21:02 Chris Aniszczyk, <caniszczyk@...> wrote:
Here's the agenda deck for tomorrow: 


We will be hearing from the keycloak project on top of welcoming Cheryl to the CNCF who will be working on ways to improve our end user ecosystem.

See everyone tomorrow!


--
Chris Aniszczyk (@cra) | +1-512-961-6719
TOC Agenda 10/2/2018

Chris Aniszczyk
 

Here's the agenda deck for tomorrow: 


We will be hearing from the keycloak project on top of welcoming Cheryl to the CNCF who will be working on ways to improve our end user ecosystem.

See everyone tomorrow!

--
Chris Aniszczyk (@cra) | +1-512-961-6719
Re: [VOTE] SAFE (Security) Working Group

Tom Keiser
 

+1 (non-binding)

On Tue, Sep 25, 2018 at 2:36 AM Zhipeng Huang <zhipengh512@...> wrote:
would like to remind TOC member to vote on WG proposition :)

On Thu, Sep 13, 2018 at 8:24 AM Sree Tummidi <stummidi@...> wrote:
+1 (non-binding)
Thanks,
Sree Tummidi


--
Zhipeng (Howard) Huang

Standard Engineer
IT Standard & Patent/IT Product Line
Huawei Technologies Co,. Ltd
Office: Huawei Industrial Base, Longgang, Shenzhen

(Previous)
Research Assistant
Mobile Ad-Hoc Network Lab, Calit2
University of California, Irvine
Office: Calit2 Building Room 2402

OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado
4881 - 4900 of 7341