Topics

Helm for Microservices Deployments


Loritsch, Berin
 

As I dive deeper into the world of Kubernetes so that I understand what I need my charts to do, and how I intend to design the templates for my environment, I am curious how Helm ties things together.

For example, we have 30+ microservices in 7 functional areas, all for one application.  We want to handle deployments with horizontal scaling, volumes, etc.

I think the picture is crystal clear when we are talking about one service.  I.e. 1 microservice would have 1 deployment, 1 hpa, ingress instructions, a service definition, and volumes and configmaps to support it.  I look at some of the examples for larger systems, and the examples I see use labels to tie things together.

When I create a parent chart that declares all of the microservices as dependencies, does Helm inject labels on my behalf?  For example, given I am in a namespace for the environment, I want to type "helm install wicketz example/wicketzapp" and then later when I need to update it or uninstall it, I would update the "wicketz" app in that custom namespace.  Additionally, as we replace microservices we no longer need because they are now redundant with Kubernetes features, I'd like the update to remove the deployments that are no longer needed.

What's the glue that binds things together?  Is that something I have to manage myself, or be aware of as I'm writing my helm charts?  Unfortunately, too many of the examples I find are too simplistic to answer the questions I'm having now.

--
Berin Loritsch

Systems Integration Lead


7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708


Paul Czarkowski <pczarkowski@...>
 

In my opinion at least, Ideally each microservice would have its own fully independent chart, and be deployed and managed independely by some sort of CI/CD solution that roughly resembles gitops.

If you need to share data or values between microservices, then you could look to using a library chart ( https://helm.sh/docs/topics/library_charts/ ) which you could use to define extra labels to help tie environments together (via metadata at least) and do things like set names for service discovery, create secrets with TLS certs, that kind of thing.

If you need to deploy all of the microservices as a bundle, then I would would reach up a layer to a tool like helmfile (https://github.com/roboll/helmfile) which can help you deploy and manage sets of loosely coupled helm charts.


From: cncf-helm@... <cncf-helm@...> on behalf of Loritsch, Berin <bloritsch@...>
Sent: Thursday, June 25, 2020 3:21 PM
To: cncf-helm@... <cncf-helm@...>
Subject: [cncf-helm] Helm for Microservices Deployments
 
As I dive deeper into the world of Kubernetes so that I understand what I need my charts to do, and how I intend to design the templates for my environment, I am curious how Helm ties things together.

For example, we have 30+ microservices in 7 functional areas, all for one application.  We want to handle deployments with horizontal scaling, volumes, etc.

I think the picture is crystal clear when we are talking about one service.  I.e. 1 microservice would have 1 deployment, 1 hpa, ingress instructions, a service definition, and volumes and configmaps to support it.  I look at some of the examples for larger systems, and the examples I see use labels to tie things together.

When I create a parent chart that declares all of the microservices as dependencies, does Helm inject labels on my behalf?  For example, given I am in a namespace for the environment, I want to type "helm install wicketz example/wicketzapp" and then later when I need to update it or uninstall it, I would update the "wicketz" app in that custom namespace.  Additionally, as we replace microservices we no longer need because they are now redundant with Kubernetes features, I'd like the update to remove the deployments that are no longer needed.

What's the glue that binds things together?  Is that something I have to manage myself, or be aware of as I'm writing my helm charts?  Unfortunately, too many of the examples I find are too simplistic to answer the questions I'm having now.

--
Berin Loritsch

Systems Integration Lead


7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708


Loritsch, Berin
 

The complication I have is that the customer I have has very outdated concepts for approving and managing deployments.  As a result, our devops pipeline is segregated from the production environment.  However as part of the delivery we can provide the containers and helm charts for all the pieces.  Once we get the go-ahead from the approval body we can run the "helm install" or "helm upgrade" process for the whole app.

It is our full intention to have sub-charts at least for each microservice should this archaic business process be brought into the 21st century.  However, in the interim we will need a master chart to define the entire application.  I did find a good approach for using labels to address some parts ot the problem: https://kubernetes.io/docs/concepts/overview/working-with-objects/common-labels/

Thanks for the pointer to helmfile.  I will look at that to see if I can use it to manage our deployments.  We manage our dev and QA instances of the application using full DevOps.  The disconnect comes in when we have to manage an acceptance test and production instance of the application on our customer's private cloud.  Each of those environments requires some customization so we can point at managed resources without risking production problems.


On Fri, Jun 26, 2020 at 11:49 AM Paul Czarkowski <pczarkowski@...> wrote:
In my opinion at least, Ideally each microservice would have its own fully independent chart, and be deployed and managed independely by some sort of CI/CD solution that roughly resembles gitops.

If you need to share data or values between microservices, then you could look to using a library chart ( https://helm.sh/docs/topics/library_charts/ ) which you could use to define extra labels to help tie environments together (via metadata at least) and do things like set names for service discovery, create secrets with TLS certs, that kind of thing.

If you need to deploy all of the microservices as a bundle, then I would would reach up a layer to a tool like helmfile (https://github.com/roboll/helmfile) which can help you deploy and manage sets of loosely coupled helm charts.

From: cncf-helm@... <cncf-helm@...> on behalf of Loritsch, Berin <bloritsch@...>
Sent: Thursday, June 25, 2020 3:21 PM
To: cncf-helm@... <cncf-helm@...>
Subject: [cncf-helm] Helm for Microservices Deployments
 
As I dive deeper into the world of Kubernetes so that I understand what I need my charts to do, and how I intend to design the templates for my environment, I am curious how Helm ties things together.

For example, we have 30+ microservices in 7 functional areas, all for one application.  We want to handle deployments with horizontal scaling, volumes, etc.

I think the picture is crystal clear when we are talking about one service.  I.e. 1 microservice would have 1 deployment, 1 hpa, ingress instructions, a service definition, and volumes and configmaps to support it.  I look at some of the examples for larger systems, and the examples I see use labels to tie things together.

When I create a parent chart that declares all of the microservices as dependencies, does Helm inject labels on my behalf?  For example, given I am in a namespace for the environment, I want to type "helm install wicketz example/wicketzapp" and then later when I need to update it or uninstall it, I would update the "wicketz" app in that custom namespace.  Additionally, as we replace microservices we no longer need because they are now redundant with Kubernetes features, I'd like the update to remove the deployments that are no longer needed.

What's the glue that binds things together?  Is that something I have to manage myself, or be aware of as I'm writing my helm charts?  Unfortunately, too many of the examples I find are too simplistic to answer the questions I'm having now.

--
Berin Loritsch

Systems Integration Lead


7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708



--
Berin Loritsch

Systems Integration Lead


7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708


Loritsch, Berin
 

Out of curiosity, why wouldn't you simply have a master helm chart for the full application that declares all of the dependencies? As opposed to learning yet another tool (https://github.com/roboll/helmfile).


On Fri, Jun 26, 2020 at 2:53 PM Loritsch, Berin via lists.cncf.io <bloritsch=novetta.com@...> wrote:
The complication I have is that the customer I have has very outdated concepts for approving and managing deployments.  As a result, our devops pipeline is segregated from the production environment.  However as part of the delivery we can provide the containers and helm charts for all the pieces.  Once we get the go-ahead from the approval body we can run the "helm install" or "helm upgrade" process for the whole app.

It is our full intention to have sub-charts at least for each microservice should this archaic business process be brought into the 21st century.  However, in the interim we will need a master chart to define the entire application.  I did find a good approach for using labels to address some parts ot the problem: https://kubernetes.io/docs/concepts/overview/working-with-objects/common-labels/

Thanks for the pointer to helmfile.  I will look at that to see if I can use it to manage our deployments.  We manage our dev and QA instances of the application using full DevOps.  The disconnect comes in when we have to manage an acceptance test and production instance of the application on our customer's private cloud.  Each of those environments requires some customization so we can point at managed resources without risking production problems.


On Fri, Jun 26, 2020 at 11:49 AM Paul Czarkowski <pczarkowski@...> wrote:
In my opinion at least, Ideally each microservice would have its own fully independent chart, and be deployed and managed independely by some sort of CI/CD solution that roughly resembles gitops.

If you need to share data or values between microservices, then you could look to using a library chart ( https://helm.sh/docs/topics/library_charts/ ) which you could use to define extra labels to help tie environments together (via metadata at least) and do things like set names for service discovery, create secrets with TLS certs, that kind of thing.

If you need to deploy all of the microservices as a bundle, then I would would reach up a layer to a tool like helmfile (https://github.com/roboll/helmfile) which can help you deploy and manage sets of loosely coupled helm charts.

From: cncf-helm@... <cncf-helm@...> on behalf of Loritsch, Berin <bloritsch@...>
Sent: Thursday, June 25, 2020 3:21 PM
To: cncf-helm@... <cncf-helm@...>
Subject: [cncf-helm] Helm for Microservices Deployments
 
As I dive deeper into the world of Kubernetes so that I understand what I need my charts to do, and how I intend to design the templates for my environment, I am curious how Helm ties things together.

For example, we have 30+ microservices in 7 functional areas, all for one application.  We want to handle deployments with horizontal scaling, volumes, etc.

I think the picture is crystal clear when we are talking about one service.  I.e. 1 microservice would have 1 deployment, 1 hpa, ingress instructions, a service definition, and volumes and configmaps to support it.  I look at some of the examples for larger systems, and the examples I see use labels to tie things together.

When I create a parent chart that declares all of the microservices as dependencies, does Helm inject labels on my behalf?  For example, given I am in a namespace for the environment, I want to type "helm install wicketz example/wicketzapp" and then later when I need to update it or uninstall it, I would update the "wicketz" app in that custom namespace.  Additionally, as we replace microservices we no longer need because they are now redundant with Kubernetes features, I'd like the update to remove the deployments that are no longer needed.

What's the glue that binds things together?  Is that something I have to manage myself, or be aware of as I'm writing my helm charts?  Unfortunately, too many of the examples I find are too simplistic to answer the questions I'm having now.

--
Berin Loritsch

Systems Integration Lead


7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708



--
Berin Loritsch

Systems Integration Lead


7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708



--
Berin Loritsch

Systems Integration Lead


7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708


Lei Zhang
 

Hi Berin

The answer should also depend on what's your intention to group these pieces together.

If the intention is to packaging and distributing, Paul's suggestion of two-layer charts should work. Another similar approach could be CNAB (https://cnab.io/) + charts, so CNAB could be used as the parent layer of your package. 

If the intention is to create a application definition for Kubernetes so you can declare dependencies of the components etc, then server side app model may be the solution, for example:  https://speakerdeck.com/resouer/oam-as-kubernetes-application-definition. In this case, you will have a helm chart to package a single app-config.yaml