Thanks for the presentation, I had a few questions and thought I'd
kick things off here.

1. Compatibility
Terminology wise, checking this is still the canonical definition from
a concept point of view?
https://kubernetes.io/docs/concepts/extend-kubernetes/operator/
I know there are several tools which help build operators
(kubebuilder, KUDO, kopf etc.), I wanted to ask about compatibility.
From a developers point of view these are all different obviously, but
what about from an end users perspective?

2. Relationship to Operator Hub
Chris also mentioned the intent to move the Operator Hub to CNCF in
https://github.com/cncf/toc/pull/303. Do you see that being under the
Operator Framework project, or under a standalone project? I think
that's interesting given the generic nature of Operators (as I
understand them above) and the Operator Framework as a specific tool.
For instance, can a kopf operator be added to Operator Hub today?

3. Relationship to Helm
As Helm is a fairly well established CNCF project, I'd be interested
in the relationship between the Operator Framework and Helm. How
involved are the Helm maintainers in the integration with the Operator
Framework?

Regarding the Operator Hub point as well. One thing I feel we should
strive to avoid plays out in my mind like:

User: Where do I find applications I can install on my Kubernetes cluster?
Helpful internet person (HIP): Why type of applications are you using?
User: ?
HIP: Helm or Operators?
User: ?
HIP: Never-mind, do to Helm Hub and Operator Hub
User: Thanks! I presume these come from two different organisations?
HIP: Oh, no, both are CNCF projects

I know their are benefits to different tools and approaches under the
CNCF banner, but when it comes to generic descriptions (Applications!)
and end user distribution it's probably worth a conversation between
multiple projects.

Thanks

Gareth

--
Gareth Rushgrove
@garethr

devopsweekly.com
garethr.dev

Hi Gareth,

sorry for the late reply - responding inline. Thanks for the questions - keep them coming!

On Wed, Oct 9, 2019 at 6:22 PM Gareth Rushgrove <gareth@...> wrote:

Thanks for the presentation, I had a few questions and thought I'd
kick things off here.

1. Compatibility
Terminology wise, checking this is still the canonical definition from
a concept point of view?
https://kubernetes.io/docs/concepts/extend-kubernetes/operator/
I know there are several tools which help build operators
(kubebuilder, KUDO, kopf etc.), I wanted to ask about compatibility.
From a developers point of view these are all different obviously, but
what about from an end users perspective?

I think this definition is still valid. An Operator is a custom controller with its own CRDs serving as the user interface (exceptions exist).

The Framework is designed to be open to the way the Operator itself is built. The packaging format and OLM don't depend on the SDK.

However there is of course good integration between the SDK and OLM.

 

2. Relationship to Operator Hub
Chris also mentioned the intent to move the Operator Hub to CNCF in
https://github.com/cncf/toc/pull/303. Do you see that being under the
Operator Framework project, or under a standalone project? I think
that's interesting given the generic nature of Operators (as I
understand them above) and the Operator Framework as a specific tool.
For instance, can a kopf operator be added to Operator Hub today?

It is under the Framework: https://github.com/operator-framework/operatorhub.io

And yes, a kopf operator, or a kubebuilder operator, or KUDO or something entirely custom can be added to this. They enabling part is the packaging format that drives the indexing and visual display.

 

3. Relationship to Helm
As Helm is a fairly well established CNCF project, I'd be interested
in the relationship between the Operator Framework and Helm. How
involved are the Helm maintainers in the integration with the Operator
Framework?

The Operator SDK project maintainers collaborate with the Helm maintainers, primarily on the Helm-based Operator pattern the SDK provides.

Helm in general focusses on Day 1 operations: bundle & install using an external tool. Operators include that and put emphasis on Day 2 operations: orchestrated updates, complex re-configuration without re-deploying from scratch, backup / restore, failover / failback, etc with kubectl. Helm serves stateless applications very well, that can be entirely maintained using the built-in functionality in k8s. Operators serve those too but provide an evolution to templating package manifests by extending the built-in k8s functionality, thus enabling k8s to run stateful applications and advanced, distributed systems.

 

Regarding the Operator Hub point as well. One thing I feel we should
strive to avoid plays out in my mind like:

User: Where do I find applications I can install on my Kubernetes cluster?
Helpful internet person (HIP): Why type of applications are you using?
User: ?
HIP: Helm or Operators?
User: ?
HIP: Never-mind, do to Helm Hub and Operator Hub
User: Thanks! I presume these come from two different organisations?
HIP: Oh, no, both are CNCF projects

I get this sentiment. I think it's important to point out that Operators and Applications are different concepts. I'd like to think of Operators as managed service providers to get managed instances of applications. So the question a user would be asking instead: do I want to enable my cluster to serve me that application and manage it or do I want to run it myself?

 

I know their are benefits to different tools and approaches under the
CNCF banner, but when it comes to generic descriptions (Applications!)
and end user distribution it's probably worth a conversation between
multiple projects.

Yes, agreed!

 

Thanks

Gareth

--
Gareth Rushgrove
@garethr

devopsweekly.com
garethr.dev

--

Daniel Messer

He / Him / His

Product Manager OpenShift

Red Hat GmbH

Wankelstrasse 5

70563 Stuttgart

dmesser@...   

On Fri, 11 Oct 2019 at 13:08, Daniel Messer <dmesser@...> wrote:

Hi Gareth,

sorry for the late reply - responding inline. Thanks for the questions - keep them coming!

On Wed, Oct 9, 2019 at 6:22 PM Gareth Rushgrove <gareth@...> wrote:

Thanks for the presentation, I had a few questions and thought I'd
kick things off here.

1. Compatibility
Terminology wise, checking this is still the canonical definition from
a concept point of view?
https://kubernetes.io/docs/concepts/extend-kubernetes/operator/
I know there are several tools which help build operators
(kubebuilder, KUDO, kopf etc.), I wanted to ask about compatibility.
From a developers point of view these are all different obviously, but
what about from an end users perspective?

I think this definition is still valid. An Operator is a custom controller with its own CRDs serving as the user interface (exceptions exist).

The Framework is designed to be open to the way the Operator itself is built. The packaging format and OLM don't depend on the SDK.

However there is of course good integration between the SDK and OLM.

 

2. Relationship to Operator Hub
Chris also mentioned the intent to move the Operator Hub to CNCF in
https://github.com/cncf/toc/pull/303. Do you see that being under the
Operator Framework project, or under a standalone project? I think
that's interesting given the generic nature of Operators (as I
understand them above) and the Operator Framework as a specific tool.
For instance, can a kopf operator be added to Operator Hub today?

It is under the Framework: https://github.com/operator-framework/operatorhub.io

And yes, a kopf operator, or a kubebuilder operator, or KUDO or something entirely custom can be added to this. They enabling part is the packaging format that drives the indexing and visual display.

 

Are there any examples of non-operator-framework operators on Operator Hub today?

 

3. Relationship to Helm
As Helm is a fairly well established CNCF project, I'd be interested
in the relationship between the Operator Framework and Helm. How
involved are the Helm maintainers in the integration with the Operator
Framework?

The Operator SDK project maintainers collaborate with the Helm maintainers, primarily on the Helm-based Operator pattern the SDK provides.

Helm in general focusses on Day 1 operations: bundle & install using an external tool. Operators include that and put emphasis on Day 2 operations: orchestrated updates, complex re-configuration without re-deploying from scratch, backup / restore, failover / failback, etc with kubectl. Helm serves stateless applications very well, that can be entirely maintained using the built-in functionality in k8s. Operators serve those too but provide an evolution to templating package manifests by extending the built-in k8s functionality, thus enabling k8s to run stateful applications and advanced, distributed systems.

 

Regarding the Operator Hub point as well. One thing I feel we should
strive to avoid plays out in my mind like:

User: Where do I find applications I can install on my Kubernetes cluster?
Helpful internet person (HIP): Why type of applications are you using?
User: ?
HIP: Helm or Operators?
User: ?
HIP: Never-mind, do to Helm Hub and Operator Hub
User: Thanks! I presume these come from two different organisations?
HIP: Oh, no, both are CNCF projects

I get this sentiment. I think it's important to point out that Operators and Applications are different concepts. I'd like to think of Operators as managed service providers to get managed instances of applications. So the question a user would be asking instead: do I want to enable my cluster to serve me that application and manage it or do I want to run it myself?

Thanks for the answers. 

I think it's fine for producers of applications to appreciate those technical different concerns, but nearly no end user is doing to ask such a complex question - they just want to run (say) MySQL.

I definitely think it would be good to talk about application stores in general in the SIG, and include folks from Helm and other operator developer tooling. I'd love to hear from maintainers of Helm and other operator tools what they think.

Gareth

 

 

I know their are benefits to different tools and approaches under the
CNCF banner, but when it comes to generic descriptions (Applications!)
and end user distribution it's probably worth a conversation between
multiple projects.

Yes, agreed!

 

Thanks

Gareth

--
Gareth Rushgrove
@garethr

devopsweekly.com
garethr.dev

--

Daniel Messer

He / Him / His

Product Manager OpenShift

Red Hat GmbH

Wankelstrasse 5

70563 Stuttgart

dmesser@...   

--

Gareth Rushgrove
@garethr

garethr.dev
devopsweekly.com

On Sat, Oct 12, 2019 at 11:24 AM Gareth Rushgrove <gareth@...> wrote:

On Fri, 11 Oct 2019 at 13:08, Daniel Messer <dmesser@...> wrote:

Hi Gareth,

sorry for the late reply - responding inline. Thanks for the questions - keep them coming!

On Wed, Oct 9, 2019 at 6:22 PM Gareth Rushgrove <gareth@...> wrote:

Thanks for the presentation, I had a few questions and thought I'd
kick things off here.

1. Compatibility
Terminology wise, checking this is still the canonical definition from
a concept point of view?
https://kubernetes.io/docs/concepts/extend-kubernetes/operator/
I know there are several tools which help build operators
(kubebuilder, KUDO, kopf etc.), I wanted to ask about compatibility.
From a developers point of view these are all different obviously, but
what about from an end users perspective?

I think this definition is still valid. An Operator is a custom controller with its own CRDs serving as the user interface (exceptions exist).

The Framework is designed to be open to the way the Operator itself is built. The packaging format and OLM don't depend on the SDK.

However there is of course good integration between the SDK and OLM.

 

2. Relationship to Operator Hub
Chris also mentioned the intent to move the Operator Hub to CNCF in
https://github.com/cncf/toc/pull/303. Do you see that being under the
Operator Framework project, or under a standalone project? I think
that's interesting given the generic nature of Operators (as I
understand them above) and the Operator Framework as a specific tool.
For instance, can a kopf operator be added to Operator Hub today?

It is under the Framework: https://github.com/operator-framework/operatorhub.io

And yes, a kopf operator, or a kubebuilder operator, or KUDO or something entirely custom can be added to this. They enabling part is the packaging format that drives the indexing and visual display.

 

Are there any examples of non-operator-framework operators on Operator Hub today?

There are lots of Operators that are not written with the SDK there. Every Operator on OperatorHub.io however is packaged for OLM to enable the visual display, versioning and install instructions.

Examples are Strimzi Kafka, Synposys, AWS Service Operator or Spark.

 

 

3. Relationship to Helm
As Helm is a fairly well established CNCF project, I'd be interested
in the relationship between the Operator Framework and Helm. How
involved are the Helm maintainers in the integration with the Operator
Framework?

The Operator SDK project maintainers collaborate with the Helm maintainers, primarily on the Helm-based Operator pattern the SDK provides.

Helm in general focusses on Day 1 operations: bundle & install using an external tool. Operators include that and put emphasis on Day 2 operations: orchestrated updates, complex re-configuration without re-deploying from scratch, backup / restore, failover / failback, etc with kubectl. Helm serves stateless applications very well, that can be entirely maintained using the built-in functionality in k8s. Operators serve those too but provide an evolution to templating package manifests by extending the built-in k8s functionality, thus enabling k8s to run stateful applications and advanced, distributed systems.

 

Regarding the Operator Hub point as well. One thing I feel we should
strive to avoid plays out in my mind like:

User: Where do I find applications I can install on my Kubernetes cluster?
Helpful internet person (HIP): Why type of applications are you using?
User: ?
HIP: Helm or Operators?
User: ?
HIP: Never-mind, do to Helm Hub and Operator Hub
User: Thanks! I presume these come from two different organisations?
HIP: Oh, no, both are CNCF projects

I get this sentiment. I think it's important to point out that Operators and Applications are different concepts. I'd like to think of Operators as managed service providers to get managed instances of applications. So the question a user would be asking instead: do I want to enable my cluster to serve me that application and manage it or do I want to run it myself?

Thanks for the answers. 

I think it's fine for producers of applications to appreciate those technical different concerns, but nearly no end user is doing to ask such a complex question - they just want to run (say) MySQL.

I think there are nuances here. For a development environment that may be true, but for someone who's responsible for running a production workload based on MySQL it's going to make a big difference whether they end up being responsible for the entire database lifecycle or can defer that to a piece of software or a service.

 

I definitely think it would be good to talk about application stores in general in the SIG, and include folks from Helm and other operator developer tooling. I'd love to hear from maintainers of Helm and other operator tools what they think.

Gareth

 

 

I know their are benefits to different tools and approaches under the
CNCF banner, but when it comes to generic descriptions (Applications!)
and end user distribution it's probably worth a conversation between
multiple projects.

Yes, agreed!

 

Thanks

Gareth

--
Gareth Rushgrove
@garethr

devopsweekly.com
garethr.dev

--

Daniel Messer

He / Him / His

Product Manager OpenShift

Red Hat GmbH

Wankelstrasse 5

70563 Stuttgart

dmesser@...   

--

Gareth Rushgrove
@garethr

garethr.dev
devopsweekly.com

--

Daniel Messer

He / Him / His

Product Manager OpenShift

Red Hat GmbH

Wankelstrasse 5

70563 Stuttgart

dmesser@...   

On Mon, 14 Oct 2019 at 16:36, Daniel Messer <dmesser@...> wrote:
>
> Agree with that Jay. The connection comes more from the OLM side since this is where the metadata format is defined that drives the OperatorHub.io index.
>

I presume OLM is Operator Lifecycle Manager?

https://github.com/operator-framework/operator-lifecycle-manager

This sounds like OperatorHub requires Operators to use OLM?

Are there any examples, or is it possible, for a Kudo (or other
operator tooling) to use OLM?

I agree with Jay, separating how apps are build from where they are
published would appear important here, otherwise we're going to have
them coupled and we'll have one app store per toolkit before we know
it.

On Mon, 2019-10-14 at 16:41 -0400, Michael Hrivnak wrote:
> The "app store" is merely the
> discovery mechanism; how many different tools will a cluster owner
> want to be involved in managing and upgrading the workloads on their
> cluster? Different cluster owners will have different tolerances, but
> many will want to standardize on one mechanism for controlling what
> versions of software are running in their cluster.

Agreed. This is why I think separating the method of discovery and
lifecycle management from the operator-sdk is critical.

Perhaps a developer would like to use the operator-sdk to develop their
Operator, but provide a non-OLM way of packaging that code. Decoupling
the SDK from the method of packaging and managing the code that is
produced by the SDK seems to enable that, no?

Please excuse my ignorance on some of this software. I have a number of questions now that I'm starting to look around.

OLM is a package manager for operators

Out of curiosity, why create another package manager? What features are missing from Helm? If some feature was missing I'd love to talk about it to learn the context and assumptions around it.

When I start to look at OLM I see things like Install Plans and Subscriptions. This looks a lot like Kubernetes Service Catalog. Is there any kind of write-up comparing the two?

You can do a lot with operators. If OLM has a lot of similarities to Service Catalog and has a bit of that intent in mind, what about the use cases aren't related to the service catalog type setup?

Should I ask these questions over on the TOC issue instead?

Thanks,

Matt

On Mon, Oct 14, 2019, at 6:03 PM, Michael Hrivnak wrote:

On Mon, Oct 14, 2019 at 4:50 PM Jay Pipes <jaypipes@...> wrote:

On Mon, 2019-10-14 at 16:41 -0400, Michael Hrivnak wrote:

> The "app store" is merely the

> discovery mechanism; how many different tools will a cluster owner

> want to be involved in managing and upgrading the workloads on their

> cluster? Different cluster owners will have different tolerances, but

> many will want to standardize on one mechanism for controlling what

> versions of software are running in their cluster.

Agreed. This is why I think separating the method of discovery and

lifecycle management from the operator-sdk is critical.

Creating an operator and creating its packaging are at least logically separate, even if one tool can do both things. When you create an operator with operator-sdk, it does not have any packaging unless you explicitly run "operator-sdk olm-catalog gen-csv" with some arguments. Adding OLM packaging is optional and does not happen by default.

 

 

Perhaps a developer would like to use the operator-sdk to develop their

Operator, but provide a non-OLM way of packaging that code. Decoupling

the SDK from the method of packaging and managing the code that is

produced by the SDK seems to enable that, no?

This is already easy to do. To start, after you've made an operator with operator-sdk, you can install it manually by running "kubectl create -f" on a few of the manifests that operator-sdk creates for you. They just contain the CRD, RBAC resources, and a Deployment. You could then turn those manifests into a helm chart that installs your operator, or use any other tool you choose.

--

Michael Hrivnak

Principal Software Engineer, RHCE 

Red Hat