Thanks for the insightful and thought-provoking blog post Bryan.  I missed the call yesterday, but co-incidentally had been noodling with similar thoughts recently, as, anecdotally, I’m also not convinced that we have the best submission review outcomes today.  I think that introducing the the partially double-blind review process would be a great step forward, and may well obviate the need for complicated per-vendor limits.

I also think that it would be super-useful for submission rejection notices to be accompanied by a few brief reviewer notes (e.g. “too much marketing pitch”, “not open source”, “previously presented”, “duplicated submission”, “off topic" etc) to help submitters to improve their chances in future (and perhaps also clarify any possible misperceptions by reviewers, as the submissions are by necessity brief). As just one illustrative data point, all 10 of my submissions to KubeCon China and US were rejected, and none of the rejections seem explainable by any of the “how to improve the odds” guidelines.  So I have no idea what to do differently in future.

Q

From: <cncf-toc@...> on behalf of Bryan Cantrill <bryan@...>
Date: Wednesday, October 3, 2018 at 12:58
To: "anthony@..." <anthony@...>
Cc: CNCF TOC <cncf-toc@...>
Subject: Re: [cncf-toc] Thoughts on KubeCon

One per vendor might be too acute, as some vendors are doing much more than others.  But having some system that limits the number of submissions per vendor (and therefore force the vendors to adopt some process to determine their best submissions) would probably help -- and would also help address the too-low acceptance rate...

        - Bryan

 

On Wed, Oct 3, 2018 at 11:54 AM Anthony Skipper <anthony@...> wrote:

I would agree with double blind.  But a max of 1 talk per vendor might also go a long way. 

On Wed, Oct 3, 2018 at 2:47 PM Bryan Cantrill <bryan@...> wrote:

On the call yesterday, Dan asked me to send out my thoughts on double-blind reviewing.  My e-mail quickly turned into a blog entry:

  http://dtrace.org/blogs/bmc/2018/10/03/should-kubecon-be-double-blind/

Something that I probably didn't highlight well enough in there is Kathryn McKinley's excellent piece on double-blind review:

  http://www.cs.utexas.edu/users/mckinley/notes/blind.html

There are certainly lots of ways to attack this problem, but I view double-blind as an essential piece -- but probably not sufficient on its own.

         - Bryan