CNCF SIG (and WG) expected deliverables


Quinton Hoole <quinton@...>
 

A quick follow-up to the discussion in today's TOC meeting regarding being clear about the TOC's expectations of deliverables from SIGs (and working groups).

Here is the discussion I had 10 months ago with the Security group regarding expectations, specifically around delivery of White Papers (github lists me as ghost, due to an unfortunate technical issue).


I think I made it very clear at the time what the TOC expected to be delivered, and the group explicitly undertook to deliver the white papers, but simply has not.

The main reason I bring this up is that I think it's important to draw a clear distinction between lack of communication from the TOC as to what's required, vs repeated lack of delivery thereof by a SIG or working group, as the solutions to the two problems are quite different.

Q

--
Quinton Hoole
quinton@...


Michael Ducy
 

The SAFE WG was an independent group (as I always understood it) that was NOT under the auspices of the CNCF or the CNCF TOC until very recently. Over those last 10 months, there's been much debate around "Categories and SIGs" and what they should deliver and that was finalized early this year. SAFE then morphed into SIG-Security a few months ago. 

So you're essentially saying, "I asked for something as a CNCF TOC member from a group that wasn't a CNCF sanctioned working group and they gave me nothing." While it would have been in the best interest of the SAFE WG to produce something, they weren't required to by any means. If I am wrong about the relationship between SAFE and the CNCF prior to them becoming SIG-Security, please correct me.

Now that being said, I do feel like the SIG-Security group should be producing white papers and the like. Specifically I'd like to see:
 - White paper of practical implementation advice
 - Cloud Native Security Landscape (This was something at SAFE had started)
 - Cloud Native Security Trail Map

This is not an exhaustive list as it doesn't included some of the Policy white papers Sarah is interested in producing. 

On Tue, Jul 2, 2019 at 12:27 PM Quinton Hoole <quinton@...> wrote:
A quick follow-up to the discussion in today's TOC meeting regarding being clear about the TOC's expectations of deliverables from SIGs (and working groups).

Here is the discussion I had 10 months ago with the Security group regarding expectations, specifically around delivery of White Papers (github lists me as ghost, due to an unfortunate technical issue).


I think I made it very clear at the time what the TOC expected to be delivered, and the group explicitly undertook to deliver the white papers, but simply has not.

The main reason I bring this up is that I think it's important to draw a clear distinction between lack of communication from the TOC as to what's required, vs repeated lack of delivery thereof by a SIG or working group, as the solutions to the two problems are quite different.

Q

--
Quinton Hoole
quinton@...


Liz Rice
 

Hi Quinton, 

This is in large part down to me! The chairs asked me for guidance on their priorities at the point where they were formally accepted as a SIG. I suggested focusing on getting some assessments done so we can get some concrete experience of this happening. To my mind this was more pressing, so we could see if/how SIGs can help the TOC scale and cope with the project assessment workload.

On 2 Jul 2019, 17:27 +0100, Quinton Hoole <quinton@...>, wrote:

A quick follow-up to the discussion in today's TOC meeting regarding being clear about the TOC's expectations of deliverables from SIGs (and working groups).

Here is the discussion I had 10 months ago with the Security group regarding expectations, specifically around delivery of White Papers (github lists me as ghost, due to an unfortunate technical issue).


I think I made it very clear at the time what the TOC expected to be delivered, and the group explicitly undertook to deliver the white papers, but simply has not.

The main reason I bring this up is that I think it's important to draw a clear distinction between lack of communication from the TOC as to what's required, vs repeated lack of delivery thereof by a SIG or working group, as the solutions to the two problems are quite different.

Q

--
Quinton Hoole
quinton@...


Quinton Hoole <quinton@...>
 

Thanks Liz

That definitely explains some of the potential confusion in the past few weeks.  But it's worth noting that the lack of delivery of the white paper dates back much further than that.  See my other message.

Q


On Tue, Jul 2, 2019 at 10:00 AM Liz Rice <liz@...> wrote:
Hi Quinton, 

This is in large part down to me! The chairs asked me for guidance on their priorities at the point where they were formally accepted as a SIG. I suggested focusing on getting some assessments done so we can get some concrete experience of this happening. To my mind this was more pressing, so we could see if/how SIGs can help the TOC scale and cope with the project assessment workload.

On 2 Jul 2019, 17:27 +0100, Quinton Hoole <quinton@...>, wrote:
A quick follow-up to the discussion in today's TOC meeting regarding being clear about the TOC's expectations of deliverables from SIGs (and working groups).

Here is the discussion I had 10 months ago with the Security group regarding expectations, specifically around delivery of White Papers (github lists me as ghost, due to an unfortunate technical issue).


I think I made it very clear at the time what the TOC expected to be delivered, and the group explicitly undertook to deliver the white papers, but simply has not.

The main reason I bring this up is that I think it's important to draw a clear distinction between lack of communication from the TOC as to what's required, vs repeated lack of delivery thereof by a SIG or working group, as the solutions to the two problems are quite different.

Q

--
Quinton Hoole
quinton@...


--
Quinton Hoole
quinton@...


Quinton Hoole <quinton@...>
 

Hi Michael

That's not accurate, or at least not my understanding, nor what's documented in the SAFE minutes.

Specifically: The minutes of 2018-04-13 say: 

Agenda:

  • NOTE: Today's session will be primarily focused on preparations for the SAFE WG proposal presentation to the CNCF TOC on 2018-4-17

  • Getting ready for CNCF TOC meeting 2018-4-17, 8am Pacific



It seems that didn't happen until Aug 2018, 3 months later.

It was specifically in that context, that I provided the following feedback and requirements from the TOC, in response to the group's request to the TOC to become a working group (the PR has subsequently been changed substantially and used to for the SIG instead, so be aware of that if you read it).  This was done in Aug and Sept 2018.


For example: Quinton: "I would like us to clearly agree upon the written proposed timeline for delivering artifacts (contained in the charter). Both the SAFE WG, and the Policy WG have been around for a year or more and to my knowledge produced very little yet in the form of concrete outputs (please correct me if I'm wrong here). So I think it is important to produce the proposed artifacts, specifically white papers, within the reasonable timeframe proposed (about a quarter per phase) starting now (i.e. Sept 2018)."

The reply from the group was:

Ultrasaurus: "I agree that dates for deliverables are helpful. We have some in our roadmap and need to fold in new deliverables from merging with Policy WG. Group will pick this up as an activity to be done in the next meeting."

One of the primary deliverables listed in the roadmap is:

Describe the landscape
Define the terminology used in the output documents, and in the community
Describe the current state (landscape) of cloud native security, ...
... common patterns in use today for system that works for cloud-native apps. For example:
Extract end-to-end view of secure access, and
Common layering or a block architecture 

It was scheduled for final delivery in "Q4 2018 - Q1 2019", but this never happened.

I also discussed this in person with Ultrasaurus at KubeCon Seatle in Dec 2018 to clarify, and she assured me at the time that plans were on track as per the above roadmap.

To be clear, this is not a finger-pointing exercise, and it's completely understandable for some target dates to be missed sometimes.
But I think it's equally important to recognise the distinction between poor communication and poor delivery.  

It seems to me that the requirements and expectations here were clearly communicated and understood, but not delivered.

Q

 





On Tue, Jul 2, 2019 at 9:57 AM Michael Ducy <michael.ducy@...> wrote:
The SAFE WG was an independent group (as I always understood it) that was NOT under the auspices of the CNCF or the CNCF TOC until very recently. Over those last 10 months, there's been much debate around "Categories and SIGs" and what they should deliver and that was finalized early this year. SAFE then morphed into SIG-Security a few months ago. 

So you're essentially saying, "I asked for something as a CNCF TOC member from a group that wasn't a CNCF sanctioned working group and they gave me nothing." While it would have been in the best interest of the SAFE WG to produce something, they weren't required to by any means. If I am wrong about the relationship between SAFE and the CNCF prior to them becoming SIG-Security, please correct me.

Now that being said, I do feel like the SIG-Security group should be producing white papers and the like. Specifically I'd like to see:
 - White paper of practical implementation advice
 - Cloud Native Security Landscape (This was something at SAFE had started)
 - Cloud Native Security Trail Map

This is not an exhaustive list as it doesn't included some of the Policy white papers Sarah is interested in producing. 

On Tue, Jul 2, 2019 at 12:27 PM Quinton Hoole <quinton@...> wrote:
A quick follow-up to the discussion in today's TOC meeting regarding being clear about the TOC's expectations of deliverables from SIGs (and working groups).

Here is the discussion I had 10 months ago with the Security group regarding expectations, specifically around delivery of White Papers (github lists me as ghost, due to an unfortunate technical issue).


I think I made it very clear at the time what the TOC expected to be delivered, and the group explicitly undertook to deliver the white papers, but simply has not.

The main reason I bring this up is that I think it's important to draw a clear distinction between lack of communication from the TOC as to what's required, vs repeated lack of delivery thereof by a SIG or working group, as the solutions to the two problems are quite different.

Q

--
Quinton Hoole
quinton@...



--
Quinton Hoole
quinton@...


Sarah Allen
 

Hi Quinton and MIchael,

You are both correct.  SAFE WG started as a group separate from CNCF.  Many early members joined the group with the assumption that the goal of the group was to become a CNCF Working Group. Other members, myself included, thought that was an interesting option, but found value in the group in any case.

Independent of org affiliation, we had planned to deliver a white paper in Q4-18.  We had also planned to publish a micro-site highlighting recordings of presentations, as well as other work streams developing landscape categories, personas and use cases.   We have delivered some documents, but first and foremost, we have formed a community where its members find the work valuable to themselves and their organizations (many of whom are CNCF member companies).  Perhaps we took on too much, certainly we did not predict the additional needs and opportunities that emerged from our prospective CNCF association.

In 2018, the TOC did not speak with one voice.  While Quinton was frustratedly waiting on the white paper, SAFE merged with the Policy WG, which brought with it a Policy White Paper already underway, along with other projects.  The TOC asked us to review in-toto and we started a process of figuring out what our role should be in assessing the security of an open source project.  That effort became the focus of the group for many months, and we heard feedback from multiple TOC members and other stakeholders that the output of this work would serve a critical need for the CNCF, as well as serving a need for many of our members, and so it was prioritized, with explicit approval from Liz, when she was appointed TOC Liaison.  

The TOCs and the SIGs need to increase transparency and communication to avoid such confusions in the future.  The TOC Liaison positions and Amye's role coordinating processes with SIGs and CNCF is a big step forward, yet there is still quite a bit of room for improvement.

Sarah



On Tue, Jul 2, 2019 at 11:09 AM Quinton Hoole <quinton@...> wrote:
Hi Michael

That's not accurate, or at least not my understanding, nor what's documented in the SAFE minutes.

Specifically: The minutes of 2018-04-13 say: 

Agenda:

  • NOTE: Today's session will be primarily focused on preparations for the SAFE WG proposal presentation to the CNCF TOC on 2018-4-17

  • Getting ready for CNCF TOC meeting 2018-4-17, 8am Pacific



It seems that didn't happen until Aug 2018, 3 months later.

It was specifically in that context, that I provided the following feedback and requirements from the TOC, in response to the group's request to the TOC to become a working group (the PR has subsequently been changed substantially and used to for the SIG instead, so be aware of that if you read it).  This was done in Aug and Sept 2018.


For example: Quinton: "I would like us to clearly agree upon the written proposed timeline for delivering artifacts (contained in the charter). Both the SAFE WG, and the Policy WG have been around for a year or more and to my knowledge produced very little yet in the form of concrete outputs (please correct me if I'm wrong here). So I think it is important to produce the proposed artifacts, specifically white papers, within the reasonable timeframe proposed (about a quarter per phase) starting now (i.e. Sept 2018)."

The reply from the group was:

Ultrasaurus: "I agree that dates for deliverables are helpful. We have some in our roadmap and need to fold in new deliverables from merging with Policy WG. Group will pick this up as an activity to be done in the next meeting."

One of the primary deliverables listed in the roadmap is:

Describe the landscape
Define the terminology used in the output documents, and in the community
Describe the current state (landscape) of cloud native security, ...
... common patterns in use today for system that works for cloud-native apps. For example:
Extract end-to-end view of secure access, and
Common layering or a block architecture 

It was scheduled for final delivery in "Q4 2018 - Q1 2019", but this never happened.

I also discussed this in person with Ultrasaurus at KubeCon Seatle in Dec 2018 to clarify, and she assured me at the time that plans were on track as per the above roadmap.

To be clear, this is not a finger-pointing exercise, and it's completely understandable for some target dates to be missed sometimes.
But I think it's equally important to recognise the distinction between poor communication and poor delivery.  

It seems to me that the requirements and expectations here were clearly communicated and understood, but not delivered.

Q

 





On Tue, Jul 2, 2019 at 9:57 AM Michael Ducy <michael.ducy@...> wrote:
The SAFE WG was an independent group (as I always understood it) that was NOT under the auspices of the CNCF or the CNCF TOC until very recently. Over those last 10 months, there's been much debate around "Categories and SIGs" and what they should deliver and that was finalized early this year. SAFE then morphed into SIG-Security a few months ago. 

So you're essentially saying, "I asked for something as a CNCF TOC member from a group that wasn't a CNCF sanctioned working group and they gave me nothing." While it would have been in the best interest of the SAFE WG to produce something, they weren't required to by any means. If I am wrong about the relationship between SAFE and the CNCF prior to them becoming SIG-Security, please correct me.

Now that being said, I do feel like the SIG-Security group should be producing white papers and the like. Specifically I'd like to see:
 - White paper of practical implementation advice
 - Cloud Native Security Landscape (This was something at SAFE had started)
 - Cloud Native Security Trail Map

This is not an exhaustive list as it doesn't included some of the Policy white papers Sarah is interested in producing. 

On Tue, Jul 2, 2019 at 12:27 PM Quinton Hoole <quinton@...> wrote:
A quick follow-up to the discussion in today's TOC meeting regarding being clear about the TOC's expectations of deliverables from SIGs (and working groups).

Here is the discussion I had 10 months ago with the Security group regarding expectations, specifically around delivery of White Papers (github lists me as ghost, due to an unfortunate technical issue).


I think I made it very clear at the time what the TOC expected to be delivered, and the group explicitly undertook to deliver the white papers, but simply has not.

The main reason I bring this up is that I think it's important to draw a clear distinction between lack of communication from the TOC as to what's required, vs repeated lack of delivery thereof by a SIG or working group, as the solutions to the two problems are quite different.

Q

--
Quinton Hoole
quinton@...



--
Quinton Hoole
quinton@...