Re: Feedback on Helm

Paul Czarkowski <pczarkowski@...>


A few things to address your template concerns.  

Firstly you can create your own chart starter packs ( which you can base subsequent charts off, this is really useful to create a sane set of defaults for similar apps in the same languages. You can also use Library Charts ( to reduce overlap/copy/paste and generally create DRYer charts.

As for "Problem is when the kubernetes template is actually rendered I have a new instance of the logging-configmap for each service, etc.  I need to be able to build and deploy my services in isolation as well as part of the whole for testing purposes."   

It's pretty common to include the dependencies in your charts so that the apps can be installed independently and start their own eureka/config-server etc, but when deployed to staging/prod you'd disable the dependencies (see condition - ) and instead have the infrastructure deployed as its own thing.  This also gives you the ability to maintain the infrastructure pieces of your application at a different lifecycle as your actual applications. You'd still use the rest of the settings for that subchart to configure your own application to talk to the upstream server.

If you have pretty complex relationships between the apps and need to be installed in a very ordered way, you can also reach towards a higher level tool like Helmfile (a declarative spec for deploying helm charts) to perform that work (Helm tries really hard not to boil the ocean). I use Helmfile extensively for this very purpose.

Also of note, Spring has some very good integrations with Kubernetes, giving you options for using kubernetes itself for service discovery and config server, which may in a homogenous environment allow you to simplify some of your app's infrastructure.  I'm far from a Spring expert, but I work pretty closely with a number of the Spring team and they're working hard to make Kubernetes a first class platform for Spring, especially when it comes to the developer experience itself.

If there's anything specific I can do to help, I'd be happy to get further into the weeds with you.

On Fri, May 1, 2020 at 2:16 PM Loritsch, Berin <bloritsch@...> wrote:
I'm in the process of modernizing our deployment to use containers and kubernetes to deploy a complex microservices application.  The current deployment uses python scripts to place things where they need to go, and while it works, that deployment model doesn't deliver on some promises we made.

We have around 30 different services we need to deploy, most of which are Spring Boot based.  The bottom line is that the deployment for individual services should be reasonably the same.  The availability check would use the Spring Actuator health endpoint, and the only a few things need to be distinct (such as service port, ingress path, resource limits, and things of that nature).

My goal was to have a very small deployment of three microservices (discovery, config, and one of the simpler non-database services).  There's some things that I like:

  • Templating makes it easier to customize deployments
  • The default template is a good 80% solution, which I was able to make some needed customizations
  • It wasn't hard to figure out how to make starters for my project
  • I can collocate the chart definition with my service source code

That said, there are a lot of rough edges:

  • You can't avoid the templating language if you customize your deployment, and if go-lang isn't something you do in your shop, it can be challenging to do otherwise simple things.
  • The templates themselves are part of the chart.  That means if I convert 15 out of 30 services to Helm, and discover something I need to fix in the generated templates, I have to change my starter and 15 services to provide that piece.
  • Helm does not have transitive package resolution, like most package management software I am used to using (Maven, Nuget, etc.)

I'm hoping I am missing something, or that there are already plans to address these rough edges.  The biggest problem is the transitive package resolution.  My helm charts look like this:
  • logging-configmap
  • system-configmap
  • eureka
    • logging-configmap
  • config-server
    • eureka
    • system-configmap
  • simple-service
    • eureka
    • config-server
The remainder of my ~30 services will be like simple-service.  In order for a successful deployment of that to happen I need the two other dependent services.  However each of those declare a copy of their dependencies, and so on.  When I run the command "helm dependency build" I will have the sub-charts loaded.

Problem is when the kubernetes template is actually rendered I have a new instance of the logging-configmap for each service, etc.  I need to be able to build and deploy my services in isolation as well as part of the whole for testing purposes.

At any rate, I would like to help make helm better.  I do have time constraints, and I'm not very familiar with go-lang.  I think this is a good start in a mostly correct direction.  It just needs to address those rough edges.

Berin Loritsch

Systems Integration Lead

7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708

Join to automatically receive all group messages.