Date   

Re: Announcing ChartCenter, the free central repository for Helm charts

Jimmy Zelinskie <jimmyzelinskie@...>
 

Congratulations on the launch, Rimas!


On Fri, Jun 26, 2020 at 9:49 AM Rimas <rmocius@...> wrote:

Helm charts make your Kubernetes deployments as easy as possible and today I’m happy to announce: ChartCenter, the free central repository for Helm charts.

 chartcenter.io is an immutable, safe, and free central repository that includes high-level vulnerability scanning, robust metadata, dependency graphs and more! Use our superior search to uncover a list of matches based on name, description, and keywords. You can also add your own charts and repositories.

You can use it as a single helm repo to access many charts from different helm repos.

Take a look to my blog for more details https://rimusz.net/chartcenter 2


Announcing ChartCenter, the free central repository for Helm charts

Rimas
 

Helm charts make your Kubernetes deployments as easy as possible and today I’m happy to announce: ChartCenter, the free central repository for Helm charts.

 chartcenter.io is an immutable, safe, and free central repository that includes high-level vulnerability scanning, robust metadata, dependency graphs and more! Use our superior search to uncover a list of matches based on name, description, and keywords. You can also add your own charts and repositories.

You can use it as a single helm repo to access many charts from different helm repos.

Take a look to my blog for more details https://rimusz.net/chartcenter 2


Helm for Microservices Deployments

Loritsch, Berin <bloritsch@...>
 

As I dive deeper into the world of Kubernetes so that I understand what I need my charts to do, and how I intend to design the templates for my environment, I am curious how Helm ties things together.

For example, we have 30+ microservices in 7 functional areas, all for one application.  We want to handle deployments with horizontal scaling, volumes, etc.

I think the picture is crystal clear when we are talking about one service.  I.e. 1 microservice would have 1 deployment, 1 hpa, ingress instructions, a service definition, and volumes and configmaps to support it.  I look at some of the examples for larger systems, and the examples I see use labels to tie things together.

When I create a parent chart that declares all of the microservices as dependencies, does Helm inject labels on my behalf?  For example, given I am in a namespace for the environment, I want to type "helm install wicketz example/wicketzapp" and then later when I need to update it or uninstall it, I would update the "wicketz" app in that custom namespace.  Additionally, as we replace microservices we no longer need because they are now redundant with Kubernetes features, I'd like the update to remove the deployments that are no longer needed.

What's the glue that binds things together?  Is that something I have to manage myself, or be aware of as I'm writing my helm charts?  Unfortunately, too many of the examples I find are too simplistic to answer the questions I'm having now.

--
Berin Loritsch

Systems Integration Lead


7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708


Helm using the same template for python and java springboot

kumaryoa@...
 

Hi,

I would like to know can we use the same helm chart template for python and java sprinboot applications. I have my git repository name(sample) is common for these both the codes. But it uses different memory configs and envrionment variables. Please suggest me how can i deploy both the code using single helm chart template.


Re: Share common configmap / data

Paul Czarkowski <pczarkowski@...>
 

Hey,

Helm Charts themselves can't access contents of a configmap.  You could either set these values as `global` values and make them available across all charts (if using chart dependencies/child charts - https://helm.sh/docs/chart_template_guide/subcharts_and_globals/ )  or you can create the configmap and then if the other charts mount that configmap as env vars or a file and load them in to each deployment that way.

On Sat, May 9, 2020 at 11:51 AM <frnd4u.ashok@...> wrote:
I'm newbie to helm and trying to understand and develop helm charts for my organization, so pardon my knowledge on the subject.This is my idea of sharing a common library configmap to access across multiple charts.
I want to identify account number and other details of AWS deployment, so I can set some values accordingly.
Is this correct? I can pass in .Values.account appropriate value and how do I go about accessing in this example account_number
apiVersion: v1
kind: ConfigMap
metadata:
name: aws
data:
account_number:
{{- if eq .Values.account "dev" }}
{{- printf "YX3XX2YX1YX6"}}
{{- else if eq .Values.account "stg" }}
{{- printf "X48XY438611X"}}
{{- else if eq .Values.account "prd" }}
{{- printf "Y83Y23X888X1"}}
{{- end }}


Re: local filesystem chart dependencies

Paul Czarkowski <pczarkowski@...>
 

Hi Marvin,

I believe the dependency entry should as follows ( `..` instead of `.` since the game-tracker chart is in the same parent directory as `teamdb-ui`:

```
dependencies:
  - name: game-tracker
    version: 0.1.0
    repository: "file://../game-tracker"
```

On Mon, May 11, 2020 at 5:36 AM Just Marvin <marvin.the.cynical.robot@...> wrote:
Hi,

    I've got two charts on my filesystem - which are not uploaded yet to any repo. They are both located in the same directory.

[zaphod@oc3027208274 helm]$ ls
game-tracker  teamdb-ui


Snippet from Charts.yaml of "teamdb-ui":

dependencies:
  - name: game-tracker
    version: 0.1.0
    repository: "file://./game-tracker"


    However, when I do a "dependency list", I get this status:

zaphod@oc3027208274 helm]$ helm dependency list teamdb-ui
NAME         VERSION REPOSITORY           STATUS
game-tracker 0.1.0   file://./game-tracker missing

    I have tried various syntaxes - file://. and other variants as well. Its also not clear if the dependency needs to be to the chart version or the app version, but I have tried with both. How do I get this working?

Regards,
Marvin


local filesystem chart dependencies

Just Marvin
 

Hi,

    I've got two charts on my filesystem - which are not uploaded yet to any repo. They are both located in the same directory.

[zaphod@oc3027208274 helm]$ ls
game-tracker  teamdb-ui


Snippet from Charts.yaml of "teamdb-ui":

dependencies:
  - name: game-tracker
    version: 0.1.0
    repository: "file://./game-tracker"


    However, when I do a "dependency list", I get this status:

zaphod@oc3027208274 helm]$ helm dependency list teamdb-ui
NAME         VERSION REPOSITORY           STATUS
game-tracker 0.1.0   file://./game-tracker missing

    I have tried various syntaxes - file://. and other variants as well. Its also not clear if the dependency needs to be to the chart version or the app version, but I have tried with both. How do I get this working?

Regards,
Marvin


Share common configmap / data

frnd4u.ashok@...
 

I'm newbie to helm and trying to understand and develop helm charts for my organization, so pardon my knowledge on the subject.This is my idea of sharing a common library configmap to access across multiple charts.
I want to identify account number and other details of AWS deployment, so I can set some values accordingly.
Is this correct? I can pass in .Values.account appropriate value and how do I go about accessing in this example account_number
apiVersion: v1
kind: ConfigMap
metadata:
name: aws
data:
account_number:
{{- if eq .Values.account "dev" }}
{{- printf "YX3XX2YX1YX6"}}
{{- else if eq .Values.account "stg" }}
{{- printf "X48XY438611X"}}
{{- else if eq .Values.account "prd" }}
{{- printf "Y83Y23X888X1"}}
{{- end }}


Re: Helm versioning and CI/CD

Fox, Kevin M <Kevin.Fox@...>
 

Yeah. versioning is always quite complicated. Distro's have had a lot more time to figure it out. If you look at the way distro's typically version things, you see something like:

mysql-5.5.50-2.el6.x86_64.rpm

The software itself has a version 5.5.50 and the packaging is a sub number off of that. '2'. What this means, is the packaging source version information can be dropped except for a release number simplifying some things.

I've been following this model for the containers I build. A container is not just the software in question, its the software + all of its dependencies + its packaging.

So, say on 01/01/2020 I build a container using Dockerfile:
FROM centos:centos7
yum install mysql-5.5.50-2

Then I build it now. The container could have all sorts of different versions of software installed. glibc for example may be significantly newer while everything else stays the same. It still is a new container and should be upgraded to.

So I fingerprint everything that goes into the container on change, and then automatically bump up the revision number if either the Dockerfile, or the Dependencies change. Ideally, the version number without the revision of the software is automatically fetchable and used in the version number as well. When done, you get a stream of images pop out like:
image: mysql-5.5.50-1
image: mysql-5.5.50-2
image: mysql-5.5.50-3
image: mysql-5.5.51-1
image: mysql-5.5.50-2
etc.

Helm charts don't really fit this model very well. You have two version numbers available. a semver Version field, and a not semver appVersion field. Users expect appVersion to somewhat fit the software being installed. A chart could have multiple container images, for example, mysql-exporter. I still use the image version above for the appVersion as its probably the closest to the users definition of appVersion. Mysql version.

For version, thats where things get ugly and I don't have it entirely flushed out. I think the algorithm is something like:
The source Version: in the chart is used as one of the inputs but isn't used for the final package directly.
The final version is kind of like a revision in a container/rpm as well as the source version. Since there is no revision in semver, I used the last value as that number.
so, X.Y.Revision
If appVersion or any of the chart dependencies, their images, or the source's Version: change, Revision is automatically incremented by one.
Ideally X.Y I think still come from the sources Version. But I'm not totally sure about that mapping. There may be dependency changes that lead to X.Y changing as well. I haven't gotten to the point of determining what the right thing to do there is.

This means releases of the chart could look like:
SourceVersion, PackageVersion, apiVersion
0.1.0 0.1.0 5.5.50-1
0.1.0 0.1.1 5.5.50-2
0.1.0 0.1.2 5.5.50-3
0.1.0 0.1.3 5.5.51-1
0.1.1 0.1.4 5.5.51-1
0.2.0 0.2.0 5.5.51-1
0.2.0 0.2.1 5.5.51-2

Helm upgrade then always does the right thing, and every version is installable.

Pretty complicated stuff. Not sure how helpful that info is, but maybe helps to explain the reasoning for the logic in that repo.

Thanks,
Kevin

________________________________________
From: cncf-helm@... <cncf-helm@...> on behalf of Loritsch, Berin <bloritsch@...>
Sent: Monday, May 4, 2020 1:22 PM
To: Matt Farina
Cc: cncf-helm@...
Subject: Re: [cncf-helm] Helm versioning and CI/CD

It does help. The main thing I'm looking for is to wrap my head around the thinking into how versioning was designed to work with helm. It's a big difference between saying "yeah that's a thing that can be done" and understanding what the ramifications of my choices are.

That being said, there are some things I want to change in the sample charts I created. I'll try to derive some inspiration from the CI definition you provided.

On Mon, May 4, 2020 at 4:02 PM Matt Farina <matt@...<mailto:matt@...>> wrote:
Berin,

I am not entirely sure of your workflow so these are just ideas and thoughts in the area.

First, appVersion is optional. You do not need to use it if you don't want to.

Second, if you are using the `helm package` command there are flags to pass in the version and appVersion.

Third, it is possible to alter things in CI. I was reminded of an example Antoine Legrand put together a few years ago. You can find it at https://gitlab.com/ant31/cookieapp/-/blob/master/.gitlab-ci.yml.<https://gitlab.com/ant31/cookieapp/-/blob/master/.gitlab-ci.yml>

The neat thing about the 3rd example is you can have a tag or even get a hash for a container image and then use that in the new chart version.

Does any of this help?

- Matt Farina

On Mon, May 4, 2020, at 1:17 PM, Loritsch, Berin wrote:
I'm in the process of writing up my recommendations for how to integrate Helm into our build and deployment infrastructure. I have a few questions that I need to understand better. The most important one has to do with how versions are intended to work with Helm.

There are two versions to worry about in any given helm file:

* Chart Version {{.Chart.version}}
* Application Version {{.Chart.appVersion}}

The way I understand it, Helm charts deal with _how_ something is deployed, and the container(s) itself deals with _what_ is deployed.

In a shop where we do continuous integration/continuous deployment we need to come up with the policies for how we manage versions. For example, we have the policy that containers are built and tagged based on the code branch or tag we are building from.

Since the chart has two version numbers embedded inside, I was wondering if there are command-line arguments to override any value. The chart itself is not going to change unless we find a defect in it. It would work with our CI/CD infrastructure if we overrode the .Chart.appVersion with the branch name when we do the full build.

If that is the case, we can build our chart as if it is production, and simply override versions when deploying specific versions of an application.

--
Berin Loritsch

Systems Integration Lead

[https://lh5.googleusercontent.com/F8VCu684Vc8-wD3bTezZ6MmgbGlabJADIpQVkdjrivp_Ey7jaQtylzhE3JxCzYYUXubtuukcibd98w8Q5PXvhnpvvg3uk_jZcrBOuXY4UItaHzhl6VusEmvfCQ3Rfdz7OeLhHOU3]<https://protect2.fireeye.com/v1/url?k=639d92c5-3f28ac0a-639db8d0-0cc47adc5e60-0b527a8768ddea91&q=1&e=97be222e-12bc-4e86-9d05-fa292b3a5f41&u=http%3A%2F%2Fwww.novetta.com%2F>

7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...<mailto:bloritsch@...>

Office (703) 735-6281

Mobile (571) 215-7708




--
Berin Loritsch

Systems Integration Lead

[https://lh5.googleusercontent.com/F8VCu684Vc8-wD3bTezZ6MmgbGlabJADIpQVkdjrivp_Ey7jaQtylzhE3JxCzYYUXubtuukcibd98w8Q5PXvhnpvvg3uk_jZcrBOuXY4UItaHzhl6VusEmvfCQ3Rfdz7OeLhHOU3]<https://protect2.fireeye.com/v1/url?k=fd3d9284-a188ac4b-fd3db891-0cc47adc5e60-d7dc2411bd30ba42&q=1&e=97be222e-12bc-4e86-9d05-fa292b3a5f41&u=http%3A%2F%2Fwww.novetta.com%2F>

7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...<mailto:bloritsch@...>

Office (703) 735-6281

Mobile (571) 215-7708


Re: Helm versioning and CI/CD

Fox, Kevin M <Kevin.Fox@...>
 

Its a lot to unpack, but in https://github.com/pnnl-miscscripts/miscscripts there are scripts to:
1. automatically build containers with newest versions of dependencies
2. apply a new version number only if the fingerprint of the software and its dependencies inside the container changes
3. push the updated images only when a new one exists.

Then it does the same for charts that point to the containers.

It essentially creates a repository that keeps everything up to date and doesn't cause spurious updates when unneeded.

It probably needs some work to be really generic. If someone were up for the challenge, that could benefit a lot of folks I think. Probably needs its own project. Currently I think its one of the biggest unsolved issues in the Cloud Native landscape. I'd be happy to contribute to such a project if there is interest in forming one.

Thanks,
Kevin

________________________________________
From: cncf-helm@... <cncf-helm@...> on behalf of Matt Farina <matt@...>
Sent: Monday, May 4, 2020 1:01 PM
To: cncf-helm@...
Subject: Re: [cncf-helm] Helm versioning and CI/CD

Berin,

I am not entirely sure of your workflow so these are just ideas and thoughts in the area.

First, appVersion is optional. You do not need to use it if you don't want to.

Second, if you are using the `helm package` command there are flags to pass in the version and appVersion.

Third, it is possible to alter things in CI. I was reminded of an example Antoine Legrand put together a few years ago. You can find it at https://gitlab.com/ant31/cookieapp/-/blob/master/.gitlab-ci.yml.<https://gitlab.com/ant31/cookieapp/-/blob/master/.gitlab-ci.yml>

The neat thing about the 3rd example is you can have a tag or even get a hash for a container image and then use that in the new chart version.

Does any of this help?

- Matt Farina

On Mon, May 4, 2020, at 1:17 PM, Loritsch, Berin wrote:
I'm in the process of writing up my recommendations for how to integrate Helm into our build and deployment infrastructure. I have a few questions that I need to understand better. The most important one has to do with how versions are intended to work with Helm.

There are two versions to worry about in any given helm file:

* Chart Version {{.Chart.version}}
* Application Version {{.Chart.appVersion}}

The way I understand it, Helm charts deal with _how_ something is deployed, and the container(s) itself deals with _what_ is deployed.

In a shop where we do continuous integration/continuous deployment we need to come up with the policies for how we manage versions. For example, we have the policy that containers are built and tagged based on the code branch or tag we are building from.

Since the chart has two version numbers embedded inside, I was wondering if there are command-line arguments to override any value. The chart itself is not going to change unless we find a defect in it. It would work with our CI/CD infrastructure if we overrode the .Chart.appVersion with the branch name when we do the full build.

If that is the case, we can build our chart as if it is production, and simply override versions when deploying specific versions of an application.

--
Berin Loritsch

Systems Integration Lead

[https://lh5.googleusercontent.com/F8VCu684Vc8-wD3bTezZ6MmgbGlabJADIpQVkdjrivp_Ey7jaQtylzhE3JxCzYYUXubtuukcibd98w8Q5PXvhnpvvg3uk_jZcrBOuXY4UItaHzhl6VusEmvfCQ3Rfdz7OeLhHOU3]<https://protect2.fireeye.com/v1/url?k=65b239c5-3907067c-65b213d0-0cc47adc5fce-fd48b812c6dd0008&q=1&e=9178ae48-7532-45cb-b0d2-c00df08d0862&u=http%3A%2F%2Fwww.novetta.com%2F>

7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...<mailto:bloritsch@...>

Office (703) 735-6281

Mobile (571) 215-7708


Re: Helm versioning and CI/CD

Loritsch, Berin <bloritsch@...>
 

It does help.  The main thing I'm looking for is to wrap my head around the thinking into how versioning was designed to work with helm.  It's a big difference between saying "yeah that's a thing that can be done" and understanding what the ramifications of my choices are.

That being said, there are some things I want to change in the sample charts I created.  I'll try to derive some inspiration from the CI definition you provided.

On Mon, May 4, 2020 at 4:02 PM Matt Farina <matt@...> wrote:
Berin,

I am not entirely sure of your workflow so these are just ideas and thoughts in the area.

First, appVersion is optional. You do not need to use it if you don't want to.

Second, if you are using the `helm package` command there are flags to pass in the version and appVersion.

Third, it is possible to alter things in CI. I was reminded of an example Antoine Legrand put together a few years ago. You can find it at https://gitlab.com/ant31/cookieapp/-/blob/master/.gitlab-ci.yml.

The neat thing about the 3rd example is you can have a tag or even get a hash for a container image and then use that in the new chart version.

Does any of this help?

- Matt Farina

On Mon, May 4, 2020, at 1:17 PM, Loritsch, Berin wrote:
I'm in the process of writing up my recommendations for how to integrate Helm into our build and deployment infrastructure.  I have a few questions that I need to understand better.  The most important one has to do with how versions are intended to work with Helm.

There are two versions to worry about in any given helm file:
  • Chart Version {{.Chart.version}}
  • Application Version {{.Chart.appVersion}}
The way I understand it, Helm charts deal with _how_ something is deployed, and the container(s) itself deals with _what_ is deployed.

In a shop where we do continuous integration/continuous deployment we need to come up with the policies for how we manage versions.  For example, we have the policy that containers are built and tagged based on the code branch or tag we are building from.

Since the chart has two version numbers embedded inside, I was wondering if there are command-line arguments to override any value.  The chart itself is not going to change unless we find a defect in it.  It would work with our CI/CD infrastructure if we overrode the .Chart.appVersion with the branch name when we do the full build.

If that is the case, we can build our chart as if it is production, and simply override versions when deploying specific versions of an application.

--
Berin Loritsch

Systems Integration Lead


7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708





--
Berin Loritsch

Systems Integration Lead


7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708


Re: Helm versioning and CI/CD

Matt Farina
 

Berin,

I am not entirely sure of your workflow so these are just ideas and thoughts in the area.

First, appVersion is optional. You do not need to use it if you don't want to.

Second, if you are using the `helm package` command there are flags to pass in the version and appVersion.

Third, it is possible to alter things in CI. I was reminded of an example Antoine Legrand put together a few years ago. You can find it at https://gitlab.com/ant31/cookieapp/-/blob/master/.gitlab-ci.yml.

The neat thing about the 3rd example is you can have a tag or even get a hash for a container image and then use that in the new chart version.

Does any of this help?

- Matt Farina

On Mon, May 4, 2020, at 1:17 PM, Loritsch, Berin wrote:
I'm in the process of writing up my recommendations for how to integrate Helm into our build and deployment infrastructure.  I have a few questions that I need to understand better.  The most important one has to do with how versions are intended to work with Helm.

There are two versions to worry about in any given helm file:
  • Chart Version {{.Chart.version}}
  • Application Version {{.Chart.appVersion}}
The way I understand it, Helm charts deal with _how_ something is deployed, and the container(s) itself deals with _what_ is deployed.

In a shop where we do continuous integration/continuous deployment we need to come up with the policies for how we manage versions.  For example, we have the policy that containers are built and tagged based on the code branch or tag we are building from.

Since the chart has two version numbers embedded inside, I was wondering if there are command-line arguments to override any value.  The chart itself is not going to change unless we find a defect in it.  It would work with our CI/CD infrastructure if we overrode the .Chart.appVersion with the branch name when we do the full build.

If that is the case, we can build our chart as if it is production, and simply override versions when deploying specific versions of an application.

--
Berin Loritsch

Systems Integration Lead


7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708




Re: Helm versioning and CI/CD

Loritsch, Berin <bloritsch@...>
 

Right, in other words, build the chart for the upcoming release, but override the values in the CI/CD pipeline if that's how you work internally.

Now, I attempted to do that using helm template directive to see what I get--and was disappointed.

$ helm template eureka --set .Chart.appVersion=java11port

Did not change appVersion for me.  I tried ".Chart.appVersion", "appVersion", "Chart.appVersion" and could not get the command line to override the appVersion for me.

I need some kind of mechanism so I can override this value, otherwise I am going to end up with hundreds of nearly identical helm charts where the only difference is the appVersion.

On Mon, May 4, 2020 at 3:22 PM <cncf@...> wrote:
The usual way app versioning in Helm works is, the chart has a value that represents which image tag the chart should install.  That tag in turn typically maps to the application version.  This doesn’t have to be true, so if a variation of it works better for you when writing your charts, feel free to use it, but for charts made publicly available, it’s generally considered a best practice and almost all of them work that way.



--
Berin Loritsch

Systems Integration Lead


7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708


Re: Helm versioning and CI/CD

Joe Thompson
 

The usual way app versioning in Helm works is, the chart has a value that represents which image tag the chart should install.  That tag in turn typically maps to the application version.  This doesn’t have to be true, so if a variation of it works better for you when writing your charts, feel free to use it, but for charts made publicly available, it’s generally considered a best practice and almost all of them work that way.


Helm versioning and CI/CD

Loritsch, Berin <bloritsch@...>
 

I'm in the process of writing up my recommendations for how to integrate Helm into our build and deployment infrastructure.  I have a few questions that I need to understand better.  The most important one has to do with how versions are intended to work with Helm.

There are two versions to worry about in any given helm file:
  • Chart Version {{.Chart.version}}
  • Application Version {{.Chart.appVersion}}
The way I understand it, Helm charts deal with _how_ something is deployed, and the container(s) itself deals with _what_ is deployed.

In a shop where we do continuous integration/continuous deployment we need to come up with the policies for how we manage versions.  For example, we have the policy that containers are built and tagged based on the code branch or tag we are building from.

Since the chart has two version numbers embedded inside, I was wondering if there are command-line arguments to override any value.  The chart itself is not going to change unless we find a defect in it.  It would work with our CI/CD infrastructure if we overrode the .Chart.appVersion with the branch name when we do the full build.

If that is the case, we can build our chart as if it is production, and simply override versions when deploying specific versions of an application.

--
Berin Loritsch

Systems Integration Lead


7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708


Re: Feedback on Helm

Loritsch, Berin <bloritsch@...>
 

Thanks for your response.  I am creating starter templates, which will help once I've got the basics nailed down.  I'm only on my second component and found things I needed to modify in the templates I customized.  Hopefully by the third one I'll be almost complete.  The library charts are something that slipped my attention.  There just isn't a lot of articles that help with the trade-offs and how to go about a decent sized microservices deployment.  One of the things that attracted me to Helm in the first place was the ability to modularize the deployment packages.

I am going to have to look into the library charts to see if they scratch the itch I have for the microservices deployment.  I'm kind of drinking from a firehose here since learning how to write charts means also learning how Kubernetes works--and most tutorials are too basic for the level of detail I need.

I am aware of the Spring integrations with Kubernetes, but with an established application I do need to take care with this lift into kubernetes infrastructure.  I fully intend to ditch Eureka and potentially Config Server in favor of services and config maps once we know what we are doing and how we want to do it.  It's hard enough getting something that is working to continue to work in a container environment.  As you said, don't boil the ocean.

The biggest questions I have right now deal with namespace management.  Helm 3 removed creating namespaces, which then makes the idea of having a hierarchical namespace harder.  For example, I would want a master namespace for each deployment (dev, int, qa, prod), but break down my microservices so they are part of a subsection.  The namespaces do allow me to set policies independently for each group of microservices.  Within "dev" I might have "dev-product1", "dev-product2", "dev-shared", "dev-frontend", "dev-data-ingest" as sub namespaces.  Do I create a helm chart to render the namespace creation?

On Fri, May 1, 2020 at 4:55 PM Paul Czarkowski <pczarkowski@...> wrote:
Berin,

A few things to address your template concerns.  

Firstly you can create your own chart starter packs (https://helm.sh/docs/topics/charts/#chart-starter-packs) which you can base subsequent charts off, this is really useful to create a sane set of defaults for similar apps in the same languages. You can also use Library Charts (https://helm.sh/docs/topics/library_charts/) to reduce overlap/copy/paste and generally create DRYer charts.

As for "Problem is when the kubernetes template is actually rendered I have a new instance of the logging-configmap for each service, etc.  I need to be able to build and deploy my services in isolation as well as part of the whole for testing purposes."   

It's pretty common to include the dependencies in your charts so that the apps can be installed independently and start their own eureka/config-server etc, but when deployed to staging/prod you'd disable the dependencies (see condition - https://helm.sh/docs/topics/charts/#tags-and-condition-fields-in-dependencies ) and instead have the infrastructure deployed as its own thing.  This also gives you the ability to maintain the infrastructure pieces of your application at a different lifecycle as your actual applications. You'd still use the rest of the settings for that subchart to configure your own application to talk to the upstream server.

If you have pretty complex relationships between the apps and need to be installed in a very ordered way, you can also reach towards a higher level tool like Helmfile (a declarative spec for deploying helm charts) to perform that work (Helm tries really hard not to boil the ocean). I use Helmfile extensively for this very purpose.

Also of note, Spring has some very good integrations with Kubernetes, giving you options for using kubernetes itself for service discovery and config server, which may in a homogenous environment allow you to simplify some of your app's infrastructure.  I'm far from a Spring expert, but I work pretty closely with a number of the Spring team and they're working hard to make Kubernetes a first class platform for Spring, especially when it comes to the developer experience itself.

If there's anything specific I can do to help, I'd be happy to get further into the weeds with you.

On Fri, May 1, 2020 at 2:16 PM Loritsch, Berin <bloritsch@...> wrote:
I'm in the process of modernizing our deployment to use containers and kubernetes to deploy a complex microservices application.  The current deployment uses python scripts to place things where they need to go, and while it works, that deployment model doesn't deliver on some promises we made.

We have around 30 different services we need to deploy, most of which are Spring Boot based.  The bottom line is that the deployment for individual services should be reasonably the same.  The availability check would use the Spring Actuator health endpoint, and the only a few things need to be distinct (such as service port, ingress path, resource limits, and things of that nature).

My goal was to have a very small deployment of three microservices (discovery, config, and one of the simpler non-database services).  There's some things that I like:

  • Templating makes it easier to customize deployments
  • The default template is a good 80% solution, which I was able to make some needed customizations
  • It wasn't hard to figure out how to make starters for my project
  • I can collocate the chart definition with my service source code

That said, there are a lot of rough edges:

  • You can't avoid the templating language if you customize your deployment, and if go-lang isn't something you do in your shop, it can be challenging to do otherwise simple things.
  • The templates themselves are part of the chart.  That means if I convert 15 out of 30 services to Helm, and discover something I need to fix in the generated templates, I have to change my starter and 15 services to provide that piece.
  • Helm does not have transitive package resolution, like most package management software I am used to using (Maven, Nuget, etc.)

I'm hoping I am missing something, or that there are already plans to address these rough edges.  The biggest problem is the transitive package resolution.  My helm charts look like this:
  • logging-configmap
  • system-configmap
  • eureka
    • logging-configmap
  • config-server
    • eureka
    • system-configmap
  • simple-service
    • eureka
    • config-server
The remainder of my ~30 services will be like simple-service.  In order for a successful deployment of that to happen I need the two other dependent services.  However each of those declare a copy of their dependencies, and so on.  When I run the command "helm dependency build" I will have the sub-charts loaded.

Problem is when the kubernetes template is actually rendered I have a new instance of the logging-configmap for each service, etc.  I need to be able to build and deploy my services in isolation as well as part of the whole for testing purposes.

At any rate, I would like to help make helm better.  I do have time constraints, and I'm not very familiar with go-lang.  I think this is a good start in a mostly correct direction.  It just needs to address those rough edges.

--
Berin Loritsch

Systems Integration Lead


7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708



--
Berin Loritsch

Systems Integration Lead


7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708


Re: Feedback on Helm

Paul Czarkowski <pczarkowski@...>
 

Berin,

A few things to address your template concerns.  

Firstly you can create your own chart starter packs (https://helm.sh/docs/topics/charts/#chart-starter-packs) which you can base subsequent charts off, this is really useful to create a sane set of defaults for similar apps in the same languages. You can also use Library Charts (https://helm.sh/docs/topics/library_charts/) to reduce overlap/copy/paste and generally create DRYer charts.

As for "Problem is when the kubernetes template is actually rendered I have a new instance of the logging-configmap for each service, etc.  I need to be able to build and deploy my services in isolation as well as part of the whole for testing purposes."   

It's pretty common to include the dependencies in your charts so that the apps can be installed independently and start their own eureka/config-server etc, but when deployed to staging/prod you'd disable the dependencies (see condition - https://helm.sh/docs/topics/charts/#tags-and-condition-fields-in-dependencies ) and instead have the infrastructure deployed as its own thing.  This also gives you the ability to maintain the infrastructure pieces of your application at a different lifecycle as your actual applications. You'd still use the rest of the settings for that subchart to configure your own application to talk to the upstream server.

If you have pretty complex relationships between the apps and need to be installed in a very ordered way, you can also reach towards a higher level tool like Helmfile (a declarative spec for deploying helm charts) to perform that work (Helm tries really hard not to boil the ocean). I use Helmfile extensively for this very purpose.

Also of note, Spring has some very good integrations with Kubernetes, giving you options for using kubernetes itself for service discovery and config server, which may in a homogenous environment allow you to simplify some of your app's infrastructure.  I'm far from a Spring expert, but I work pretty closely with a number of the Spring team and they're working hard to make Kubernetes a first class platform for Spring, especially when it comes to the developer experience itself.

If there's anything specific I can do to help, I'd be happy to get further into the weeds with you.

On Fri, May 1, 2020 at 2:16 PM Loritsch, Berin <bloritsch@...> wrote:
I'm in the process of modernizing our deployment to use containers and kubernetes to deploy a complex microservices application.  The current deployment uses python scripts to place things where they need to go, and while it works, that deployment model doesn't deliver on some promises we made.

We have around 30 different services we need to deploy, most of which are Spring Boot based.  The bottom line is that the deployment for individual services should be reasonably the same.  The availability check would use the Spring Actuator health endpoint, and the only a few things need to be distinct (such as service port, ingress path, resource limits, and things of that nature).

My goal was to have a very small deployment of three microservices (discovery, config, and one of the simpler non-database services).  There's some things that I like:

  • Templating makes it easier to customize deployments
  • The default template is a good 80% solution, which I was able to make some needed customizations
  • It wasn't hard to figure out how to make starters for my project
  • I can collocate the chart definition with my service source code

That said, there are a lot of rough edges:

  • You can't avoid the templating language if you customize your deployment, and if go-lang isn't something you do in your shop, it can be challenging to do otherwise simple things.
  • The templates themselves are part of the chart.  That means if I convert 15 out of 30 services to Helm, and discover something I need to fix in the generated templates, I have to change my starter and 15 services to provide that piece.
  • Helm does not have transitive package resolution, like most package management software I am used to using (Maven, Nuget, etc.)

I'm hoping I am missing something, or that there are already plans to address these rough edges.  The biggest problem is the transitive package resolution.  My helm charts look like this:
  • logging-configmap
  • system-configmap
  • eureka
    • logging-configmap
  • config-server
    • eureka
    • system-configmap
  • simple-service
    • eureka
    • config-server
The remainder of my ~30 services will be like simple-service.  In order for a successful deployment of that to happen I need the two other dependent services.  However each of those declare a copy of their dependencies, and so on.  When I run the command "helm dependency build" I will have the sub-charts loaded.

Problem is when the kubernetes template is actually rendered I have a new instance of the logging-configmap for each service, etc.  I need to be able to build and deploy my services in isolation as well as part of the whole for testing purposes.

At any rate, I would like to help make helm better.  I do have time constraints, and I'm not very familiar with go-lang.  I think this is a good start in a mostly correct direction.  It just needs to address those rough edges.

--
Berin Loritsch

Systems Integration Lead


7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708


Re: Feedback on Helm

Matt Farina
 

Berin,

Thanks for such an extensive write-up. I can't address all of it now but I wanted to talk about transitive dependencies. I am curious what you are looking for in respect to that as far as handling goes.

When you have a tree of dependencies you can't exactly flatten them. For example, if you have a dependency on WordPress and Drupal and they both have a dependency on MySQL you can't flatten the tree to have one MySQL. You will end up with two as they are operating as services, possible with very different configurations, for each of those. So, the transitive handling is mildly different from something like a programming language package manager.

When `helm package` is run (to create the archive) the dependencies are bundled in the `charts` directory. These are the direct dependencies. Each of the direct dependencies dependencies would be bundled in its own `charts` directory and so on.

This is why I ask about your use case. I wonder what's going on. I'd like to better understand it and the usage pattern.

Cheers,
Matt Farina

On Fri, May 1, 2020, at 3:16 PM, Loritsch, Berin wrote:
I'm in the process of modernizing our deployment to use containers and kubernetes to deploy a complex microservices application.  The current deployment uses python scripts to place things where they need to go, and while it works, that deployment model doesn't deliver on some promises we made.

We have around 30 different services we need to deploy, most of which are Spring Boot based.  The bottom line is that the deployment for individual services should be reasonably the same.  The availability check would use the Spring Actuator health endpoint, and the only a few things need to be distinct (such as service port, ingress path, resource limits, and things of that nature).

My goal was to have a very small deployment of three microservices (discovery, config, and one of the simpler non-database services).  There's some things that I like:

  • Templating makes it easier to customize deployments
  • The default template is a good 80% solution, which I was able to make some needed customizations
  • It wasn't hard to figure out how to make starters for my project
  • I can collocate the chart definition with my service source code

That said, there are a lot of rough edges:

  • You can't avoid the templating language if you customize your deployment, and if go-lang isn't something you do in your shop, it can be challenging to do otherwise simple things.
  • The templates themselves are part of the chart.  That means if I convert 15 out of 30 services to Helm, and discover something I need to fix in the generated templates, I have to change my starter and 15 services to provide that piece.
  • Helm does not have transitive package resolution, like most package management software I am used to using (Maven, Nuget, etc.)

I'm hoping I am missing something, or that there are already plans to address these rough edges.  The biggest problem is the transitive package resolution.  My helm charts look like this:
  • logging-configmap
  • system-configmap
  • eureka
    • logging-configmap
  • config-server
    • eureka
    • system-configmap
  • simple-service
    • eureka
    • config-server
The remainder of my ~30 services will be like simple-service.  In order for a successful deployment of that to happen I need the two other dependent services.  However each of those declare a copy of their dependencies, and so on.  When I run the command "helm dependency build" I will have the sub-charts loaded.

Problem is when the kubernetes template is actually rendered I have a new instance of the logging-configmap for each service, etc.  I need to be able to build and deploy my services in isolation as well as part of the whole for testing purposes.

At any rate, I would like to help make helm better.  I do have time constraints, and I'm not very familiar with go-lang.  I think this is a good start in a mostly correct direction.  It just needs to address those rough edges.

--
Berin Loritsch

Systems Integration Lead


7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708




Feedback on Helm

bloritsch@...
 

I'm in the process of modernizing our deployment to use containers and kubernetes to deploy a complex microservices application.  The current deployment uses python scripts to place things where they need to go, and while it works, that deployment model doesn't deliver on some promises we made.

We have around 30 different services we need to deploy, most of which are Spring Boot based.  The bottom line is that the deployment for individual services should be reasonably the same.  The availability check would use the Spring Actuator health endpoint, and the only a few things need to be distinct (such as service port, ingress path, resource limits, and things of that nature).

My goal was to have a very small deployment of three microservices (discovery, config, and one of the simpler non-database services).  There's some things that I like:

  • Templating makes it easier to customize deployments
  • The default template is a good 80% solution, which I was able to make some needed customizations
  • It wasn't hard to figure out how to make starters for my project
  • I can collocate the chart definition with my service source code

That said, there are a lot of rough edges:

  • You can't avoid the templating language if you customize your deployment, and if go-lang isn't something you do in your shop, it can be challenging to do otherwise simple things.
  • The templates themselves are part of the chart.  That means if I convert 15 out of 30 services to Helm, and discover something I need to fix in the generated templates, I have to change my starter and 15 services to provide that piece.
  • Helm does not have transitive package resolution, like most package management software I am used to using (Maven, Nuget, etc.)

I'm hoping I am missing something, or that there are already plans to address these rough edges.  The biggest problem is the transitive package resolution.  My helm charts look like this:
  • logging-configmap
  • system-configmap
  • eureka
    • logging-configmap
  • config-server
    • eureka
    • system-configmap
  • simple-service
    • eureka
    • config-server
The remainder of my ~30 services will be like simple-service.  In order for a successful deployment of that to happen I need the two other dependent services.  However each of those declare a copy of their dependencies, and so on.  When I run the command "helm dependency build" I will have the sub-charts loaded.

Problem is when the kubernetes template is actually rendered I have a new instance of the logging-configmap for each service, etc.  I need to be able to build and deploy my services in isolation as well as part of the whole for testing purposes.

At any rate, I would like to help make helm better.  I do have time constraints, and I'm not very familiar with go-lang.  I think this is a good start in a mostly correct direction.  It just needs to address those rough edges.

--
Berin Loritsch

Systems Integration Lead


7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708


Re: localizing Helm docs [feedback requested]

Matt Farina
 

Thanks for suggesting the Kubernetes process. I'll take a look at it.

I just took Gitlocalize for a spin. I had a couple problems with it (i.e., hugo front matter handling and DCO). Won't work for us. Though, I can see how it makes many cases much easier. Translations all through a web ui that end up as PRs. Both human and machine translations supported. It's almost there but not quite.

- Matt Farina

On Mon, Mar 30, 2020, at 3:51 PM, Jim Angel wrote:
Happy to chat about how k8s docs is doing it if you want!

Most of the process is outlined here: https://kubernetes.io/docs/contribute/localization/

I would echo Chris' statement that it works out decently but there are challenges...


On Mon, Mar 30, 2020, 14:47 Chris Aniszczyk <caniszczyk@...> wrote:
Fedora and OpenStack use http://zanata.org 

It's notable the Kubernetes community just treats translations as code and that has served them decently, but they have invested in doing public "doc sprints" to help on board and train folks.

On Mon, Mar 30, 2020 at 2:42 PM Matt Farina <matt@...> wrote:

I'm kinda interested in the human based tools more than automating translations. From what I've been told, the automation quality is so-so.

But, we have people interested in translating docs. But, our process for doing so is developer focused and non-devs who want to do translations will have a hard time. I'm wondering if there's a way to ease their flow. I was reminded of the simple experience Drupal had and was wondering if any other tools make it that easy.

On Mon, Mar 30, 2020, at 3:38 PM, Matt Fisher wrote:
Something I was interested in looking into was the Microsoft Translator API. From its description,

> [...] that developers can easily integrate into their applications websites, tools, or any solution requiring multi-language support such as website localization, e-commerce, customer support, messaging applications, internal communication, and more.

I have not tried this program yet, but we could potentially use our Azure sponsorship money to consume this service and provide localization support for the website.

I'd be most curious to see how it'd handle syntax and code blocks.






1504220459230_microsoft.png

Matthew Fisher

Caffeinated Software Engineer

Microsoft Canada




From: cncf-helm@... <cncf-helm@...> on behalf of Matt Farina via Lists.Cncf.Io <matt=mattfarina.com@...>
Sent: Monday, March 30, 2020 12:01 PM
To: cncf-helm@... <cncf-helm@...>
Cc: cncf-helm@... <cncf-helm@...>
Subject: [cncf-helm] localizing Helm docs [feedback requested]
 
The Helm website and docs are setup for localization / i18n. The flow of contributing translations is the same as contributing docs and code.

This has left me wondering... should Helm use some outside tooling to make the translation process easier? Is there something you're familiar with and would suggest? To seed the conversation, I was looking at https://gitlocalize.com/.

Do you have any tips or pointers? We want to make the process easier. Especially for non-devs.

- Matt Farina

Attachments:
  • Outlook-1504220459.png





--
Chris Aniszczyk (@cra) | +1-512-961-6719




Attachments:
  • Outlook-1504220459.png

141 - 160 of 457