|
Re: full yaml/json list of all Strimzi configurations?
Jakub Scholz
Hi Anthony, I do not think we have any files like that. It often doesn't make sense to use all the different options or it is even impossible. So we do not maintain any such files. The documentation has the API reference where you can find all the options: https://strimzi.io/docs/operators/latest/using.html#schema_properties Thanks & Regards Jakub On Sun, May 9, 2021 at 2:30 PM Anthony Percy <anthcp@...> wrote: Hi there, |
|
|
|
full yaml/json list of all Strimzi configurations?
Anthony Percy
Hi there,
I am looking for a full list of yaml/json Strimzi configurations, does one exist? This could just be a large YAML or Json file showing all strimzi objects and options or a json schema file etc... regards Anthony |
|
|
|
RC1 of Strimzi Operators 0.23.0
Jakub Scholz
Release Candidate 1 of Strimzi Operators 0.23.0 is now available for testing with a lot of changes and improvements. This release supports only the API version v1beta2 and CRD version apiextensions.k8s.io/v1. Migration to v1beta2 needs to be completed for all Strimzi CRDs and CRs before the upgrade to 0.23 is done! For more details about the CRD upgrades, see the documentation. The main changes since the 0.22 release include: * Add support for Kafka 2.8.0 and 2.6.2, remove support for Kafka 2.5.x * Make it possible to configure maximum number of connections and maximum connection creation rate in listener configuration * Add support for configuring finalizers for loadbalancer type listeners * Use dedicated Service Account for Kafka Connect Build on Kubernetes * Remove direct ZooKeeper access for handling user quotas in the User Operator. Add usage of Admin Client API instead. * Migrate to CRD v1 (required by Kubernetes 1.22+) * Support for configuring custom Authorizer implementation * Changed Reconciliation interval for Topic Operator from 90 to 120 seconds (to keep it the same as for other operators) * Changed Zookeeper session timeout default value to 18 seconds for Topic and User Operators (for improved resiliency) * Removed requirement for replicas and partitions KafkaTopic spec making these parameters optional * Support to configure a custom filter for parent CR's labels propagation into subresources * Allow disabling service links (environment variables describing Kubernetes services) in Pod template * Update Kaniko executor to 1.6.0 * Add support for separate control plane listener (disabled by default, available via the ControlPlaneListener feature gate) * Support for Dual Stack networking There are also several deprecations and removals. For more details and installation files, go to https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.23.0-rc1 Any feedback can be provided on the Strimzi mailing list, on the #strimzi Slack channel on CNCF Slack or as a GitHub issue. Thanks & Regards Jakub & Strimzi team |
|
|
|
[ANNOUNCE] [RELEASE] Strimzi Kafka OAuth library 0.7.2 released
Jakub Scholz
Hi, Version 0.7.2 of the Strimzi Kafka OAuth library is now available: https://github.com/strimzi/strimzi-kafka-oauth/releases/tag/0.7.2. The main change since 0.7.1 is an improvement to OAuth over SASL-PLAIN: * Introduced 'no-client-credentials' mode with OAuth over PLAIN (#107) For more details about the new features see the RELEASE_NOTES and the README files. All changes can be found under the 0.7.2 milestone. Thanks to everyone who contributed to this release! Thanks & Regards Strimzi team |
|
|
|
[ANNOUNCE] [RELEASE] Strimzi Kafka Operators 0.22.1 released
Jakub Scholz
Strimzi Operators 0.22.1 which addresses the known issues from 0.22.0 release is now available: * Do not use ownerReference for Entity Operator role in separate watched namespace (#4588) * Minor documentation and system test improvements For more details, see the 0.22.1 release on GitHub. See the 0.22.0 release for information about CRD upgrades, deprecations and removals. Thanks to everyone who contributed to this release! |
|
|
|
[ANNOUNCE] [RELEASE] Strimzi Kafka Operators 0.22.0 released
Jakub Scholz
Strimzi Operators 0.22.0 has been released with a lot of changes and improvements. This release introduces new API version `v1beta2` to all Strimzi custom resources. This is a preparation for migration to `apiextensions/v1` which is needed because Kubernetes 1.22 will remove support for `apiextensions/v1beta1`. Migration to `v1beta2` needs to be completed for all Strimzi CRDs and CRs after the upgrade to 0.22 is done and before upgrading to Strimzi 0.23 which will support only Strimzi `v1beta2` APIs and `apiextensions/v1` CRDs. For more details about the CRD upgrades, see the documentation: https://strimzi.io/docs/operators/0.22.0/deploying.html#assembly-upgrade-resources-str The main changes since the 0.21 release include: * Add `v1beta2` version for all resources. `v1beta2` removes all deprecated fields. * Add annotations that enable the operator to restart Kafka Connect connectors or tasks. The annotations can be applied to the KafkaConnector and the KafkaMirrorMaker2 custom resources. * Add additional configuration options for the Kaniko executor used by the Kafka Connect Build on Kubernetes * Add support for JMX options configuration of all Kafka Connect (KC, KC2SI, MM2) * Update Strimzi Kafka OAuth to version 0.7 and add support for new features: * OAuth authentication over SASL PLAIN mechanism * Checking token audience * Validating tokens using JSONPath filter queries to perform custom checks * Fix Cruise Control crash loop when updating container configurations * Configure external logging `ConfigMap` name and key. * Add support for configuring labels and annotations in ClusterRoleBindings created as part of Kafka and Kafka Connect clusters * Add support for Ingress v1 in Kubernetes 1.19 and newer * Add support for Kafka 2.6.1 * List topics used by a Kafka Connect connector in the `.status` section of the `KafkaConnector` custom resource * Bump Cruise Control to v2.5.37 for Kafka 2.7 support. Note this new version of Cruise Control uses `Log4j 2` and is supported by dynamic logging configuration (where logging properties are defined in a ConfigMap). However, existing `Log4j` configurations must be updated to `Log4j 2` configurations. * Support pausing reconciliation of CR with annotation `strimzi.io/pause-reconciliation` There are also several deprecations and removals and one known issue. For more details and installation files, go to https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.22.0 Thanks to everyone who contributed to this release! |
|
|
|
RC1 of Strimzi Operators 0.22.0
Jakub Scholz
Release Candidate 1 of Strimzi Operators 0.22.0 is now available for testing with a lot of changes and improvements. This release introduces new API version `v1beta2` to all Strimzi custom resources. This is a preparation for migration to `apiextensions/v1` which is needed because Kubernetes 1.22 will remove support for `apiextensions/v1beta1`. Migration to `v1beta2` needs to be completed for all Strimzi CRDs and CRs after the upgrade to 0.22 is done and before upgrading to Strimzi 0.23 which will support only Strimzi `v1beta2` APIs and `apiextensions/v1` CRDs. For more details about the CRD upgrades, see the documentation: https://strimzi.io/docs/operators/master/deploying.html#assembly-upgrade-resources-str The main changes since the 0.21 release include: * Add `v1beta2` version for all resources. `v1beta2` removes all deprecated fields. * Add annotations that enable the operator to restart Kafka Connect connectors or tasks. The annotations can be applied to the KafkaConnector and the KafkaMirrorMaker2 custom resources. * Add additional configuration options for the Kaniko executor used by the Kafka Connect Build on Kubernetes * Add support for JMX options configuration of all Kafka Connect (KC, KC2SI, MM2) * Update Strimzi Kafka OAuth to version 0.7 and add support for new features: * OAuth authentication over SASL PLAIN mechanism * Checking token audience * Validating tokens using JSONPath filter queries to perform custom checks * Fix Cruise Control crash loop when updating container configurations * Configure external logging `ConfigMap` name and key. * Add support for configuring labels and annotations in ClusterRoleBindings created as part of Kafka and Kafka Connect clusters * Add support for Ingress v1 in Kubernetes 1.19 and newer * Add support for Kafka 2.6.1 * List topics used by a Kafka Connect connector in the `.status` section of the `KafkaConnector` custom resource * Bump Cruise Control to v2.5.37 for Kafka 2.7 support. Note this new version of Cruise Control uses `Log4j 2` and is supported by dynamic logging configuration (where logging properties are defined in a ConfigMap). However, existing `Log4j` configurations must be updated to `Log4j 2` configurations. * Support pausing reconciliation of CR with annotation `strimzi.io/pause-reconciliation` There are also several deprecations and removals. For more details and installation files, go to https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.22.0-rc1 Any feedback can be provided on the Strimzi mailing list, on the #strimzi Slack channel on CNCF Slack or as a GitHub issue. Thanks & Regards Jakub & Strimzi team |
|
|
|
[ANNOUNCE] [RELEASE] Strimzi Kafka OAuth library 0.7.1 released
Jakub Scholz
Hi, Version 0.7.1 of the Strimzi Kafka OAuth library is now available: https://github.com/strimzi/strimzi-kafka-oauth/releases/tag/0.7.1. The main changes since 0.7.0 are two bugfixes: * Fixed OAuth over PLAIN intermittent failures (#95) * Fix NPE in Keycloak Authorizer (#97) For more details about the new features see the RELEASE_NOTES and the README files. All changes can be found under the 0.7.1 milestone. Thanks to everyone who contributed to this release! Thanks & Regards Strimzi team |
|
|
|
[ANNOUNCE] [RELEASE] Strimzi Kafka OAuth library 0.7.0 released
Jakub Scholz
Hi, Version 0.7.0 of the Strimzi Kafka OAuth library is now available: https://github.com/strimzi/strimzi-kafka-oauth/releases/tag/0.7.0. The main changes since 0.6.x are: * OAuth authentication over SASL PLAIN * Checking `audience` of the JWT token in the server part of the OAuth library * Custom claim checking For more details about the new features see the RELEASE_NOTES and the README files. All changes can be found under the 0.7.0 milestone. Thanks to everyone who contributed to this release! Thanks & Regards Strimzi team |
|
|
|
RC2 of Strimzi Kafka OAuth library 0.7.0
Jakub Scholz
Hi, Release Candidate 2 of the 0.7.0 version of the Strimzi Kafka OAuth library is now available for testing: https://github.com/strimzi/strimzi-kafka-oauth/releases/tag/0.7.0-rc2. Compared to RC1, it adds the custom claim checking feature and test improvements. The main changes since 0.6.x are: * OAuth authentication over SASL PLAIN * Checking `audience` of the JWT token in the server part of the OAuth library * Custom claim checking To test it, you can use the staging Maven repository: <repositories> <repository> <id>staging</id> <url>https://oss.sonatype.org/content/repositories/iostrimzi-1090</url> </repository> </repositories> For more details about the new features see the RELEASE_NOTES and the README files. All changes can be found under the 0.7.0 milestone. Any feedback can be provided on the mailing list, on Slack or as a GitHub issue. Thanks & Regards Jakub |
|
|
|
Re: Kafka clients are getting disconnected frequently
Jakub Scholz
As I said on Slack ... I wonder about two things ... * What client does it use? From the log, it does not seem to be like the official Java client. Might be worth trying it (for example with some dummy data) with the official client to eliminate any possible client issues and get a log which is easier to understand. * Is the connection from outside or inside Kubernetes? If it is from outside, it might be worth trying it from inside and see if that makes any difference. Jakub On Wed, Jan 20, 2021 at 10:53 AM Majid Ma <majid.ma@...> wrote: Hello Strimizi Users, |
|
|
|
Kafka clients are getting disconnected frequently
Majid Ma
Hello Strimizi Users,
We have deployed kafka using strmizi operator v0.20.1 and we see the kafka clients are getting disconnected frequently, though we don't see any stability issues in the applications using kafka. We see the same behaviour in the other environments where we have deployed kafka using the strimzi operator v0.20.0 We don't see this issue for the environments that does not uses kafka deployed using strimizi operator. The following is our application logs, If you see there are continuous connection failures. 2021-01-20 05:55:36,651 - WARNING - AA - kafka.client - --- - Node 2 connection failed -- refreshing metadata
2021-01-20 05:55:36,944 - WARNING - AA - kafka.client - --- - Node 0 connection failed -- refreshing metadata
2021-01-20 05:55:39,944 - WARNING - AA - kafka.client - --- - Node 0 connection failed -- refreshing metadata
2021-01-20 05:56:07,464 - WARNING - AA - kafka.client - --- - Node 1 connection failed -- refreshing metadata
2021-01-20 05:56:12,944 - WARNING - AA - kafka.client - --- - Node 0 connection failed -- refreshing metadata
2021-01-20 05:56:13,227 - WARNING - AA - kafka.client - --- - Node 2 connection failed -- refreshing metadata
2021-01-20 05:56:13,384 - WARNING - AA - kafka.client - --- - Node 1 connection failed -- refreshing metadata
2021-01-20 05:56:14,943 - WARNING - AA - kafka.client - --- - Node 0 connection failed -- refreshing metadata Any thoughts or inputs/suggestions will be appreciated. |
|
|
|
[ANNOUNCE] [RELEASE] Strimzi Kafka Operators 0.21.1 released
Jakub Scholz
Shortly after releasing 0.21.0 we discovered two bugs affecting it. That is why we now released Strimzi Operators 0.21.1 with the following bug-fixes: * Fix broken links in the OAuth documentation (#4265) * Fix the network-policies handling when metrics config from CM is used (#4261) For more details and installation files, go to https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.21.1 Thanks & Regards Strimzi team |
|
|
|
Re: [cncf-strimzi-dev] [ANNOUNCE] [RELEASE] Strimzi Kafka Operators 0.21.0 released
Jakub Scholz
We found a bug in the new feature for configuring metrics from ConfigMap instead of directly in the custom resource. When used, it in some cases does not properly configure the network policies for the port 9404 used for the metrics. As a workaround, either keep using the old configuration or create manually an additional network policy with your custom name for the port 9404. This bug will be fixed in 0.21.1. Thanks & Regards Jakub
|
|
|
|
[ANNOUNCE] [RELEASE] Strimzi Kafka Operators 0.21.0 released
Jakub Scholz
Strimzi Operators 0.21.0 is now available with a lot of important changes and improvements. The main changes since the 0.20.0 release include: * Add support for Kafka 2.7.0 * Add support for declarative management of connector plugins in Kafka Connect CR * Add `inter.broker.protocol.version` to the default configuration in example YAMLs * Add support for `secretPrefix` property for User Operator to prefix all secret names created from KafkaUser resource. * Allow configuring labels and annotations for Cluster CA certificate secrets * Add the JAAS configuration string in the sasl.jaas.config property to the generated secrets for KafkaUser with SCRAM-SHA-512 authentication. * Strimzi `test-container` has been renamed to `strimzi-test-container` to make the name more clear * Updated the CPU usage metric in the Kafka, ZooKeeper and Cruise Control dashboards to include the CPU kernel time (other than the current user time) * Allow disabling ownerReference on CA secrets * Make it possible to run Strimzi operators and operands with read-only root filesystem * Move from Docker Hub to Quay.io as our container registry * Add possibility to configure DeploymentStrategy for Kafka Connect, Kafka Mirror Maker (1 and 2), and Kafka Bridge * Support passing metrics configuration as an external ConfigMap * Enable CORS configuration for Cruise Control * Add support for rolling individual Kafka or ZooKeeper pods through the Cluster Operator using an annotation * Add support for Topology Spread Constraints in Pod templates * Make Kafka `cluster-id` (KIP-78) available on Kafka CRD status There are also some deprecations you should be aware of: * The `metrics` field in the Strimzi custom resources has been deprecated and will be removed in the future. For configuring metrics, use the new `metricsConfig` field and pass the configuration via ConfigMap. Since 0.21.0 Strimzi supports Kubernetes version 1.16 and newer. For more details and installation files, go to https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.21.0 Many thanks to everyone who contributed to this release! Thanks & Regards Strimzi team |
|
|
|
RC1 of Strimzi Operators 0.21.0
Jakub Scholz
Release Candidate 1 of Strimzi Operators 0.21.0 is now available for testing with a lot of changes and improvements. The main changes since the 0.20.0 release include: * Add support for declarative management of connector plugins in Kafka Connect CR * Add `inter.broker.protocol.version` to the default configuration in example YAMLs * Add support for `secretPrefix` property for User Operator to prefix all secret names created from KafkaUser resource. * Allow configuring labels and annotations for Cluster CA certificate secrets * Add the JAAS configuration string in the sasl.jaas.config property to the generated secrets for KafkaUser with SCRAM-SHA-512 authentication. * Strimzi `test-container` has been renamed to `strimzi-test-container` to make the name more clear * Updated the CPU usage metric in the Kafka, ZooKeeper and Cruise Control dashboards to include the CPU kernel time (other than the current user time) * Allow disabling ownerReference on CA secrets * Make it possible to run Strimzi operators and operands with read-only root filesystem * Move from Docker Hub to Quay.io as our container registry * Add possibility to configure DeploymentStrategy for Kafka Connect, Kafka Mirror Maker (1 and 2), and Kafka Bridge * Support passing metrics configuration as an external ConfigMap * Enable CORS configuration for Cruise Control * Add support for rolling individual Kafka or ZooKeeper pods through the Cluster Operator using an annotation * Add support for Topology Spread Constraints in Pod templates * Make Kafka `cluster-id` (KIP-78) available on Kafka CRD status There are also some deprecations and removals which you should be aware of: * The `metrics` field in the Strimzi custom resources has been deprecated and will be removed in the future. For configuring metrics, use the new `metricsConfig` field and pass the configuration via ConfigMap. This version of Strimzi supports Kubernetes 1.16 and higher. For more details and installation files, go to https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.21.0-rc1 Any feedback can be provided on the Strimzi mailing list, on the #strimzi Slack channel on CNCF Slack or as a GitHub issue. Thanks & Regards Jakub & Strimzi team |
|
|
|
[ANNOUNCE] [RELEASE] Strimzi Kafka Operators 0.20.1 released
Jakub Scholz
Strimzi Operators 0.20.1 is now available with several bug-fixes: * Silent error from missing CRB RBAC in Kafka Connect when not needed (#4019) * Fine-tune the Kafka Exporter health checks (#3885) * Correct connect configuration comparison (#3987) * Avoid changing custom resource status because of HashSet ordering (#4069) * Fix the client rack-awareness in Kafka Connect (#3903) * Connect default logging not expanded (#4057) * Topic operator improvements (#3982) * Do not use ownerReference in UO and TO bindings into a different namespace (#4080) * Remove owner references from ClusterRoleBindings (#4077) * Topic Operator metrics (#3883) For more details and installation files, go to https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.20.1 Many thanks to everyone who contributed to this release! Thanks & Regards Strimzi team |
|
|
|
RC1 of Strimzi Operators 0.20.1
Jakub Scholz
Release Candidate 1 of Strimzi Operators 0.20.1 is now available for testing with several bug-fixes: * Silent error from missing CRB RBAC in Kafka Connect when not needed (#4019) * Fine-tune the Kafka Exporter health checks (#3885) * Correct connect configuration comparison (#3987) * Avoid changing custom resource status because of HashSet ordering (#4069) * Fix the client rack-awareness in Kafka Connect (#3903) * Connect default logging not expanded (#4057) * Topic operator improvements (#3982) * Do not use ownerReference in UO and TO bindings into a different namespace (#4080) * Remove owner references from ClusterRoleBindings (#4077) * Topic Operator metrics (#3883) For more details and installation files, go to https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.20.1-rc1 Any feedback can be provided on the Strimzi mailing list, on the #strimzi Slack channel on CNCF Slack or as a GitHub issue. Thanks & Regards Jakub |
|
|
|
RC1 of Strimzi Kafka OAuth library 0.7.0
Jakub Scholz
Hi, Release Candidate 1 of the 0.7.0 version of the Strimzi Kafka OAuth library is now available for testing: https://github.com/strimzi/strimzi-kafka-oauth/releases/tag/0.7.0-rc1. The main changes are: * OAuth authentication over SASL PLAIN * Checking `audience` of the JWT token in the server part of the OAuth library To test it, you can use the staging Maven repository: <repositories> <repository> <id>staging</id> <url>https://oss.sonatype.org/content/repositories/iostrimzi-1084</url> </repository> </repositories> Any feedback can be provided on the mailing list, on Slack or as a GitHub issue. Thanks & Regards Jakub |
|
|
|
[ANNOUNCE] [RELEASE] Strimzi Kafka Operators 0.20.0 released
Jakub Scholz
Strimzi Operators 0.20.0 is now available with a lot of important changes and improvements. The main changes since the 0.19.0 release include: * Add support for Kafka 2.5.1 and 2.6.0. Remove support for 2.4.0 and 2.4.1 * Make `listeners` configurable as an array and add support for more different listeners in single cluster * Remove TLS sidecars from Kafka pods => Kafka now uses native TLS to connect to ZooKeeper * Updated to Cruise Control 2.5.11, which adds Kafka 2.6.0 support and fixes a previous issue with CPU utilization statistics for containers. As a result, the CPUCapacityGoal has now been enabled. * Configure Cluster Operator logging using ConfigMap instead of environment variable and support dynamic changes * Switch to use the `AclAuthorizer` class for the `simple` Kafka authorization type. `AclAuthorizer` contains new features such as the ability to control the amount of authorization logs in the broker logs. * Support dynamic logging configuration of Kafka, Kafka Connect, Kafka Connect S2I and Kafka MirrorMaker2 * Add support for `client.rack` property for Kafka Connect to use `fetch from closest replica` feature. * Add support for configuring `hostAliases` in Pod templates * Cruise Control metrics integration including a new Grafana dashboard * Improved operators Grafana dashboard * Improvements to OAuth authentication and Keycloak authorization There are also some deprecations and removals which you should be aware of: * Deprecation of the old listener configuration * Removal of monitoring port on Kafka and ZooKeeper related services * Deprecation of Kafka TLS sidecar configuration 0.20.0 will also be the last version of Strimzi that will support Kubernetes 1.11-1.15. Future versions will support Kubernetes 1.16+. For more details and installation files, go to https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.20.0 Many thanks to everyone who contributed to this release! Thanks & Regards Strimzi team |
|
|