Date   

FYI, Helm Hub to Artifact Hub

Matt Farina
 

The Helm project has decided to deprecate the Helm Hub and redirect people to the Artifact Hub. We have been tracking this work for some time.

We are now at the point where the redirect will go into place, docs will go up, and a blog announcement will be doing out. This will be happening soon. This is a heads up that it's coming soon.

Regards,
Matt Farina


Small governance change

Matt Farina
 

I wanted to share that we have made a small governance change to Helm. There was a section in the governance that pertained to the initial election of org maintainers. That event happened a couple years ago. It is not relevant to ongoing or future work and we have git which records the history of the governance.

So, that section was removed following a vote.


Note, because it was the removal of no-longer relevant governance (the equivalent of dead code) I didn't email it out. If it were something that would make a relevant change I would have emailed the list about it.

Regards,
Matt Farina


Re: Why do we need Helm Tests? What does they solve?

@Tejaswini_Vadlamudi
 

@Matt:  I am trying to understand different test cases that can be tested via the helm test. I now agree that we need these tests but I was unsure how can help/aid my chart consumers. Most of the helm charts today in helm hub are testing connections to their ports. I was thinking if we can use "helm tests" to notice if the deployment was successful or not i.e. to understand if microservices are deployed properly as per deployment configuration and ready to serve traffic. Am I thinking correctly? Can we use helm tests to test readiness and liveness checks of microservices or is it a bad idea (rely on monitoring instead)?

@Berin: Thanks for the suggestions. I had a quick look at Terratest. If we have helm tests, we can either run "helm test" or "ct lint". I think Terratest is a go library support for writing test code and also offers support for deployment, validation of charts, and infrastructure readiness. Does Terratest run "helm test" as part of its validation or does that something else?

Thanks,
Teja


Re: Why do we need Helm Tests? What does they solve?

Loritsch, Berin <bloritsch@...>
 

Keep in mind that there is a tool called Terratest https://github.com/gruntwork-io/terratest that is designed to test your deployments.  It works with a wide array of deployment tools, including Kubernetes.  Might be worth seeing what they provide.


On Wed, Sep 9, 2020 at 7:37 PM <vtejaswini1@...> wrote:
Thank you Matt, you always share prompt and correct reflections!! I'm very happy with the above explanation, sounds reasonable. :)
I agree that chart consumers (automation/human beings) are different actors compared to chart owners/developers. If chart owners don't involve in the chart consumer's helm deployments and tests, chart owners of course need to provide some sort of sanity tests.

Best Regards,
Teja 



--
Berin Loritsch

Systems Integration Lead


7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708


Re: Why do we need Helm Tests? What does they solve?

@Tejaswini_Vadlamudi
 

Thank you Matt, you always share prompt and correct reflections!! I'm very happy with the above explanation, sounds reasonable. :)
I agree that chart consumers (automation/human beings) are different actors compared to chart owners/developers. If chart owners don't involve in the chart consumer's helm deployments and tests, chart owners of course need to provide some sort of sanity tests.

Best Regards,
Teja 


Re: Why do we need Helm Tests? What does they solve?

Matt Farina
 

Teja,

This is a good question.

There are a couple things that come to mind for uses of Helm tests.

Helm tests enable a chart author to package tests in a chart and ship them to a chart consumer. The chart consumer (sometimes called the Helm user) can then install a chart and test that it's running. This is useful as Helm is a package manager and the one creating the chart is often not the one installing and using the chart.

Helm is used across many versions of Kubernetes. That includes supported and unsupported versions. When a chart is packaged up by a chart creator it may be tested against the version of Kubernetes they use but does that mean it works against the version of Kubernetes a chart user is dealing with? Testing passed to the users in a common way provides a means to help figure it out.

One tool to help with testing is the Chart Testing tool (https://github.com/helm/chart-testing). It's used for the CI testing in the stable charts repo.

Regards,
Matt


On Mon, Sep 7, 2020, at 9:11 AM, vtejaswini1@... wrote:
Hi,
I want to understand why we need helm tests? I have my CI automation which deploys all of my charts and checks if the feature is working or not.
I don't understand why I need "helm tests" in the helm chart directories. Also, is it the only way-forward or do we have any other tools that can help us for the time being?

Thanks,
Teja


Why do we need Helm Tests? What does they solve?

@Tejaswini_Vadlamudi
 

Hi,
I want to understand why we need helm tests? I have my CI automation which deploys all of my charts and checks if the feature is working or not.
I don't understand why I need "helm tests" in the helm chart directories. Also, is it the only way-forward or do we have any other tools that can help us for the time being?

Thanks,
Teja


Stable and Incubator chart repo deprecation

Matt Farina
 

This has been shared previously but we've come to realize that not everyone has noticed.

On November 13th of this year the stable and incubator chart repos will be marked obsolete and will be going away.

There are two elements to this. The first is that they will be obsolete and we will no longer be accepting changes to them. Any charts that want to be around long term will need to migrate to a new location. If you have questions on doing this or running your own repository, we are happy to help. We have documented how to run a repo on GitHub and have released tools, like the chart releaser, to help.

Those charts can be made discoverable through the Helm Hub and Artifact Hub. The Artifact Hub is listed because the Helm Hub is migrating there. You can find the migration issue at https://github.com/helm/hub/issues/439.

The other element is that the chart repositories will be going away. While the git repository will still be available the Helm repository will be garbage collected and no longer available. This can happen on November 13th or any day after that. I would plan on Nov 13th to be on the safe side.

jFrog Chart Center does have a cache of the charts. I have been told this will remain after the stable and incubator repos are gone.

If you have any questions about this please let us know. This was initially announced months ago but I realize the message has been lost with everything else going on. We are putting a renewed focus on it in an attempt to make a clean of a transition as possible.

Regards,
Matt Farina


Helm Hub move to Artifact Hub

Matt Farina
 

The Helm Hub has made it far easier to find charts that are distributed around the Internet. General purpose search engines don't make it easy and the Helm Hub, being Kubernetes and charts specific, provided an improvement.

The Helm Hub is built on Monocular which was originally designed for purposes other than running a large many repo site. It has worked well but its limitations have started to showcase themselves. For example, the repository listing on the charts search page does not visually scale well and the pageload times increase with each new chart added to the site. We knew that some changes needed to happen for the Helm Hub to scale better.

The Artifact Hub is a Kubernetes artifact search site. It handles Helm charts and other cloud native artifacts. This site is self service for both publishers and consumers and designed to scale. The Artifact Hub was recently accepted as a CNCF Sandbox project and was born out of work in the CNCF.

Instead of investing in Helm Hub software that does a better job scaling, we are going to migrate over to use the Artifact Hub. This means the Helm Hub site will eventually redirect to the Artifact Hub and the Helm Hub codebase will be archived.

The issue tracking this migration is at https://github.com/helm/hub/issues/439

Some steps in the migration have already begun. This includes making all of the existing Helm Hub repositories available in the Artifact Hub. Those that were not already listed on the Artifact Hub were migrated. If you own one of the repositories that has been migrated and would like to take over management please file an issue on the Helm Hub and we can transfer the ownership.

We are happy to answer any questions related to this.


Feedback on proposal

jorgecarleitao@...
 

Hi,

I am writing to formalize a proposal to extend Helm 3 in a backward compatible manner to allow passing derived values to subcharts without exposing them in `values.yaml`. I wrote this an issue on github, 8576, and opened a PR with a minimal implementation the proposal, 8580. I consider this a major feature, and as such I am opening this topic here to gather your thoughts.

I am willing to drive this, but I will need some guidance.

Best,
Jorge


Re: stepping down

Martin Hickey
 

Hi Michelle,
 
It was great to have had a chance to work with you. All the best in your new adventures.

Regards,
Martin
 
 

----- Original message -----
From: "minooral via lists.cncf.io" <minooral=microsoft.com@...>
Sent by: cncf-helm@...
To: cncf-helm@...
Cc:
Subject: [EXTERNAL] [cncf-helm] stepping down
Date: Wed, Jul 8, 2020 12:35 AM
 

[Edited Message Follows]

Hey Helmettes,

It's been a pleasure being a core maintainer on the Helm project since it's initial release Helm Classic (v1) in 2015 and probably the coolest experience of my career. I am officially stepping down as core maintainer to focus on other responsibilities. I've learned a lot from everyone involved and appreciate having had the opportunity to work with you all.
https://github.com/helm/helm/pull/8414

--- Feel free to ignore the rest of this if you don't care for long messages but I do want to take a few minutes to give my thanks.---

Thank you Matt Butcher for giving me room and guidance to build out parts of Helm v1 and v2. I learned a lot about how to do open source and build a community, how to research and think through design decisions and iterate on them, and how to write Go from you. I will always appreciate your patience and the opportunities you encouraged me to take. I remember you asking me to compile my code on the spot and and show Bitnami (and others) what we were working on in the early days. Obviously, I don't recommend that for everyone but when you asked me do that (more than once), it helped me learn how to talk about what we were working on on the spot. I learned how to demo, give the right amount of context without prep, and it made me more confident about showing work in progress. Thanks for helping build a safe and respectful environment to work in (you and Adam both). Thank you Adam Reese for being my partner in crime, for always supporting me and having my back. I loved going through this journey with you from building a PaaS at Engine Yard to building a package manager for Kubernetes. I will never forget the times where we'd get on calls and admit to each other we were both totally lost lol. I learned a lot about how to ask the right questions and articulate problems more clearly from you. Thank you Brian Hardock for helping me think deeper about the software and tools we use. Adnan, Miguel, Sameer, Jack Francis, Brian Grant, Vic, Sebastian, Bridget, Matt Fisher, Martin. I've had a blast with you all. I know I am missing people (sorry). Super proud of this tool and community and still humbled I got the opportunity to work on it.


Cheers,

Michelle Noorali
 


stepping down

minooral@...
 
Edited

Hey Helmettes,

It's been a pleasure being a core maintainer on the Helm project since it's initial release Helm Classic (v1) in 2015 and probably the coolest experience of my career. I am officially stepping down as core maintainer to focus on other responsibilities. I've learned a lot from everyone involved and appreciate having had the opportunity to work with you all.
https://github.com/helm/helm/pull/8414

--- Feel free to ignore the rest of this if you don't care for long messages but I do want to take a few minutes to give my thanks.---

Thank you Matt Butcher for giving me room and guidance to build out parts of Helm v1 and v2. I learned a lot about how to do open source and build a community, how to research and think through design decisions and iterate on them, and how to write Go from you. I will always appreciate your patience and the opportunities you encouraged me to take. I remember you asking me to compile my code on the spot and and show Bitnami (and others) what we were working on in the early days. Obviously, I don't recommend that for everyone but when you asked me do that (more than once), it helped me learn how to talk about what we were working on on the spot. I learned how to demo, give the right amount of context without prep, and it made me more confident about showing work in progress. Thanks for helping build a safe and respectful environment to work in (you and Adam both). Thank you Adam Reese for being my partner in crime, for always supporting me and having my back. I loved going through this journey with you from building a PaaS at Engine Yard to building a package manager for Kubernetes. I will never forget the times where we'd get on calls and admit to each other we were both totally lost lol. I learned a lot about how to ask the right questions and articulate problems more clearly from you. Thank you Brian Hardock for helping me think deeper about the software and tools we use. Adnan, Miguel, Sameer, Jack Francis, Brian Grant, Vic, Sebastian, Bridget, Matt Fisher, Martin. I've had a blast with you all. I know I am missing people (sorry). Super proud of this tool and community and still humbled I got the opportunity to work on it.


Cheers,

Michelle Noorali


Re: How can we achieve automatic rollback in case of "helm upgrade" failures?

Matt Fisher <matt.fisher@...>
 

Not sure where you got the information that the atomic flag doesn’t exist in Helm 3. I just checked, and it’s been there since Helm 3.0.0, and it’s present in HEAD.



If you want to automatically roll back in case of a failure, you can use the atomic flag. I’m not aware of another feature to handle rollbacks.

1504220459230_microsoft.png

Matthew Fisher

Caffeinated Software Engineer

Microsoft Canada



From: cncf-helm@... <cncf-helm@...> on behalf of vtejaswini1 via lists.cncf.io <vtejaswini1=gmail.com@...>
Sent: Wednesday, July 1, 2020 10:00 AM
To: cncf-helm@... <cncf-helm@...>
Subject: [cncf-helm] How can we achieve automatic rollback in case of "helm upgrade" failures?
 
Hi,

Does anyone have ideas about supporting automatic rollback in terms of failures? Over slack, people recommend different sets of tools. What is the recommended approach or best feasible approach to use?

Do I need to use any helm operator or?

I also notice helm3 not supporting the atomic flag, which was created to support automatic rollback.

Thanks,
Teja


How can we achieve automatic rollback in case of "helm upgrade" failures?

@Tejaswini_Vadlamudi
 

Hi,

Does anyone have ideas about supporting automatic rollback in terms of failures? Over slack, people recommend different sets of tools. What is the recommended approach or best feasible approach to use?

Do I need to use any helm operator or?

I also notice helm3 not supporting the atomic flag, which was created to support automatic rollback.

Thanks,
Teja


Re: What is the proper way to rollback using helm, Is it using "helm rollback" or "helm upgrade olderversion"?

Paul Czarkowski <pczarkowski@...>
 

Hi Teja!

Both are completely valid option.  People generally like the rollback commands as it's rolling your kubernetes manifests back to a known good version, therefore if there was an error introduced by a change in the Helm charts themselves it would not continue that error.

However this option doesn't take into account the application itself. If there is a bug in the application or you have a backwards incompatible database schema change, then it would make sense to roll forward with a newer image and a `helm upgrade`.



From: cncf-helm@... <cncf-helm@...> on behalf of vtejaswini1@... <vtejaswini1@...>
Sent: Wednesday, July 1, 2020 8:35 AM
To: cncf-helm@... <cncf-helm@...>
Subject: [cncf-helm] What is the proper way to rollback using helm, Is it using "helm rollback" or "helm upgrade olderversion"?
 
Hi,

I was looking at downgrading software on failure. I wonder if I need to run upgrade an older version or use the "helm rollback" command.

Well, I don't have enough arguments about why I need to use "helm rollback" for rolling back software.

Does anyone know the reasons behind this support?

Thanks,
Teja


What is the proper way to rollback using helm, Is it using "helm rollback" or "helm upgrade olderversion"?

@Tejaswini_Vadlamudi
 

Hi,

I was looking at downgrading software on failure. I wonder if I need to run upgrade an older version or use the "helm rollback" command.

Well, I don't have enough arguments about why I need to use "helm rollback" for rolling back software.

Does anyone know the reasons behind this support?

Thanks,
Teja


Re: Helm for Microservices Deployments

Lei Zhang
 

Hi Berin

The answer should also depend on what's your intention to group these pieces together.

If the intention is to packaging and distributing, Paul's suggestion of two-layer charts should work. Another similar approach could be CNAB (https://cnab.io/) + charts, so CNAB could be used as the parent layer of your package. 

If the intention is to create a application definition for Kubernetes so you can declare dependencies of the components etc, then server side app model may be the solution, for example:  https://speakerdeck.com/resouer/oam-as-kubernetes-application-definition. In this case, you will have a helm chart to package a single app-config.yaml


Re: Helm for Microservices Deployments

Loritsch, Berin <bloritsch@...>
 

Out of curiosity, why wouldn't you simply have a master helm chart for the full application that declares all of the dependencies? As opposed to learning yet another tool (https://github.com/roboll/helmfile).


On Fri, Jun 26, 2020 at 2:53 PM Loritsch, Berin via lists.cncf.io <bloritsch=novetta.com@...> wrote:
The complication I have is that the customer I have has very outdated concepts for approving and managing deployments.  As a result, our devops pipeline is segregated from the production environment.  However as part of the delivery we can provide the containers and helm charts for all the pieces.  Once we get the go-ahead from the approval body we can run the "helm install" or "helm upgrade" process for the whole app.

It is our full intention to have sub-charts at least for each microservice should this archaic business process be brought into the 21st century.  However, in the interim we will need a master chart to define the entire application.  I did find a good approach for using labels to address some parts ot the problem: https://kubernetes.io/docs/concepts/overview/working-with-objects/common-labels/

Thanks for the pointer to helmfile.  I will look at that to see if I can use it to manage our deployments.  We manage our dev and QA instances of the application using full DevOps.  The disconnect comes in when we have to manage an acceptance test and production instance of the application on our customer's private cloud.  Each of those environments requires some customization so we can point at managed resources without risking production problems.


On Fri, Jun 26, 2020 at 11:49 AM Paul Czarkowski <pczarkowski@...> wrote:
In my opinion at least, Ideally each microservice would have its own fully independent chart, and be deployed and managed independely by some sort of CI/CD solution that roughly resembles gitops.

If you need to share data or values between microservices, then you could look to using a library chart ( https://helm.sh/docs/topics/library_charts/ ) which you could use to define extra labels to help tie environments together (via metadata at least) and do things like set names for service discovery, create secrets with TLS certs, that kind of thing.

If you need to deploy all of the microservices as a bundle, then I would would reach up a layer to a tool like helmfile (https://github.com/roboll/helmfile) which can help you deploy and manage sets of loosely coupled helm charts.

From: cncf-helm@... <cncf-helm@...> on behalf of Loritsch, Berin <bloritsch@...>
Sent: Thursday, June 25, 2020 3:21 PM
To: cncf-helm@... <cncf-helm@...>
Subject: [cncf-helm] Helm for Microservices Deployments
 
As I dive deeper into the world of Kubernetes so that I understand what I need my charts to do, and how I intend to design the templates for my environment, I am curious how Helm ties things together.

For example, we have 30+ microservices in 7 functional areas, all for one application.  We want to handle deployments with horizontal scaling, volumes, etc.

I think the picture is crystal clear when we are talking about one service.  I.e. 1 microservice would have 1 deployment, 1 hpa, ingress instructions, a service definition, and volumes and configmaps to support it.  I look at some of the examples for larger systems, and the examples I see use labels to tie things together.

When I create a parent chart that declares all of the microservices as dependencies, does Helm inject labels on my behalf?  For example, given I am in a namespace for the environment, I want to type "helm install wicketz example/wicketzapp" and then later when I need to update it or uninstall it, I would update the "wicketz" app in that custom namespace.  Additionally, as we replace microservices we no longer need because they are now redundant with Kubernetes features, I'd like the update to remove the deployments that are no longer needed.

What's the glue that binds things together?  Is that something I have to manage myself, or be aware of as I'm writing my helm charts?  Unfortunately, too many of the examples I find are too simplistic to answer the questions I'm having now.

--
Berin Loritsch

Systems Integration Lead


7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708



--
Berin Loritsch

Systems Integration Lead


7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708



--
Berin Loritsch

Systems Integration Lead


7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708


Re: Helm for Microservices Deployments

Loritsch, Berin <bloritsch@...>
 

The complication I have is that the customer I have has very outdated concepts for approving and managing deployments.  As a result, our devops pipeline is segregated from the production environment.  However as part of the delivery we can provide the containers and helm charts for all the pieces.  Once we get the go-ahead from the approval body we can run the "helm install" or "helm upgrade" process for the whole app.

It is our full intention to have sub-charts at least for each microservice should this archaic business process be brought into the 21st century.  However, in the interim we will need a master chart to define the entire application.  I did find a good approach for using labels to address some parts ot the problem: https://kubernetes.io/docs/concepts/overview/working-with-objects/common-labels/

Thanks for the pointer to helmfile.  I will look at that to see if I can use it to manage our deployments.  We manage our dev and QA instances of the application using full DevOps.  The disconnect comes in when we have to manage an acceptance test and production instance of the application on our customer's private cloud.  Each of those environments requires some customization so we can point at managed resources without risking production problems.


On Fri, Jun 26, 2020 at 11:49 AM Paul Czarkowski <pczarkowski@...> wrote:
In my opinion at least, Ideally each microservice would have its own fully independent chart, and be deployed and managed independely by some sort of CI/CD solution that roughly resembles gitops.

If you need to share data or values between microservices, then you could look to using a library chart ( https://helm.sh/docs/topics/library_charts/ ) which you could use to define extra labels to help tie environments together (via metadata at least) and do things like set names for service discovery, create secrets with TLS certs, that kind of thing.

If you need to deploy all of the microservices as a bundle, then I would would reach up a layer to a tool like helmfile (https://github.com/roboll/helmfile) which can help you deploy and manage sets of loosely coupled helm charts.

From: cncf-helm@... <cncf-helm@...> on behalf of Loritsch, Berin <bloritsch@...>
Sent: Thursday, June 25, 2020 3:21 PM
To: cncf-helm@... <cncf-helm@...>
Subject: [cncf-helm] Helm for Microservices Deployments
 
As I dive deeper into the world of Kubernetes so that I understand what I need my charts to do, and how I intend to design the templates for my environment, I am curious how Helm ties things together.

For example, we have 30+ microservices in 7 functional areas, all for one application.  We want to handle deployments with horizontal scaling, volumes, etc.

I think the picture is crystal clear when we are talking about one service.  I.e. 1 microservice would have 1 deployment, 1 hpa, ingress instructions, a service definition, and volumes and configmaps to support it.  I look at some of the examples for larger systems, and the examples I see use labels to tie things together.

When I create a parent chart that declares all of the microservices as dependencies, does Helm inject labels on my behalf?  For example, given I am in a namespace for the environment, I want to type "helm install wicketz example/wicketzapp" and then later when I need to update it or uninstall it, I would update the "wicketz" app in that custom namespace.  Additionally, as we replace microservices we no longer need because they are now redundant with Kubernetes features, I'd like the update to remove the deployments that are no longer needed.

What's the glue that binds things together?  Is that something I have to manage myself, or be aware of as I'm writing my helm charts?  Unfortunately, too many of the examples I find are too simplistic to answer the questions I'm having now.

--
Berin Loritsch

Systems Integration Lead


7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708



--
Berin Loritsch

Systems Integration Lead


7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708


Re: Helm for Microservices Deployments

Paul Czarkowski <pczarkowski@...>
 

In my opinion at least, Ideally each microservice would have its own fully independent chart, and be deployed and managed independely by some sort of CI/CD solution that roughly resembles gitops.

If you need to share data or values between microservices, then you could look to using a library chart ( https://helm.sh/docs/topics/library_charts/ ) which you could use to define extra labels to help tie environments together (via metadata at least) and do things like set names for service discovery, create secrets with TLS certs, that kind of thing.

If you need to deploy all of the microservices as a bundle, then I would would reach up a layer to a tool like helmfile (https://github.com/roboll/helmfile) which can help you deploy and manage sets of loosely coupled helm charts.


From: cncf-helm@... <cncf-helm@...> on behalf of Loritsch, Berin <bloritsch@...>
Sent: Thursday, June 25, 2020 3:21 PM
To: cncf-helm@... <cncf-helm@...>
Subject: [cncf-helm] Helm for Microservices Deployments
 
As I dive deeper into the world of Kubernetes so that I understand what I need my charts to do, and how I intend to design the templates for my environment, I am curious how Helm ties things together.

For example, we have 30+ microservices in 7 functional areas, all for one application.  We want to handle deployments with horizontal scaling, volumes, etc.

I think the picture is crystal clear when we are talking about one service.  I.e. 1 microservice would have 1 deployment, 1 hpa, ingress instructions, a service definition, and volumes and configmaps to support it.  I look at some of the examples for larger systems, and the examples I see use labels to tie things together.

When I create a parent chart that declares all of the microservices as dependencies, does Helm inject labels on my behalf?  For example, given I am in a namespace for the environment, I want to type "helm install wicketz example/wicketzapp" and then later when I need to update it or uninstall it, I would update the "wicketz" app in that custom namespace.  Additionally, as we replace microservices we no longer need because they are now redundant with Kubernetes features, I'd like the update to remove the deployments that are no longer needed.

What's the glue that binds things together?  Is that something I have to manage myself, or be aware of as I'm writing my helm charts?  Unfortunately, too many of the examples I find are too simplistic to answer the questions I'm having now.

--
Berin Loritsch

Systems Integration Lead


7921 Jones Branch Drive
McLean, VA 22102
Email bloritsch@...

Office (703) 735-6281

Mobile (571) 215-7708

121 - 140 of 457