|
Re: FW: Questions about setting up kafka-http bridge
Jakub Scholz
Hi Hamze, Thanks & Regards Jakub On Tue, Apr 6, 2021 at 8:48 PM Hamza Aslam <hamza.aslam@...> wrote:
|
|
|
|
FW: Questions about setting up kafka-http bridge
Hamza Aslam <hamza.aslam@...>
I was able to start it up. My postman keeps giving me a 415 format not supported error how do we fix that?
From: Hamza Aslam <hamza.aslam@...>
Sent: Tuesday, April 6, 2021 11:18 AM To: cncf-strimzi-dev@... Subject: Questions about setting up kafka-http bridge
Hi
I am trying to set up the kafka-http bridge so I can read from and write to a topic. I had the kafka-connect running which I halted and ran the bridge but the get requests say that the server isn’t up even though I ran the bridge and it did not give me any errors and looks like its running. On github it says to edit the applications file but I don’t think anything needs to be changed or updated there. What am I missing. The documentation on the website doesn’t have step by step instructions which I am afraid I will need
Kind regards Hamza
|
|
|
|
Questions about setting up kafka-http bridge
Hamza Aslam <hamza.aslam@...>
Hi
I am trying to set up the kafka-http bridge so I can read from and write to a topic. I had the kafka-connect running which I halted and ran the bridge but the get requests say that the server isn’t up even though I ran the bridge and it did not give me any errors and looks like its running. On github it says to edit the applications file but I don’t think anything needs to be changed or updated there. What am I missing. The documentation on the website doesn’t have step by step instructions which I am afraid I will need
Kind regards Hamza
|
|
|
|
[ANNOUNCE] [RELEASE] Strimzi Kafka Operators 0.22.1 released
Jakub Scholz
Strimzi Operators 0.22.1 which addresses the known issues from 0.22.0 release is now available: * Do not use ownerReference for Entity Operator role in separate watched namespace (#4588) * Minor documentation and system test improvements For more details, see the 0.22.1 release on GitHub. See the 0.22.0 release for information about CRD upgrades, deprecations and removals. Thanks to everyone who contributed to this release! |
|
|
|
[ANNOUNCE] [RELEASE] Strimzi Kafka Operators 0.22.0 released
Jakub Scholz
Strimzi Operators 0.22.0 has been released with a lot of changes and improvements. This release introduces new API version `v1beta2` to all Strimzi custom resources. This is a preparation for migration to `apiextensions/v1` which is needed because Kubernetes 1.22 will remove support for `apiextensions/v1beta1`. Migration to `v1beta2` needs to be completed for all Strimzi CRDs and CRs after the upgrade to 0.22 is done and before upgrading to Strimzi 0.23 which will support only Strimzi `v1beta2` APIs and `apiextensions/v1` CRDs. For more details about the CRD upgrades, see the documentation: https://strimzi.io/docs/operators/0.22.0/deploying.html#assembly-upgrade-resources-str The main changes since the 0.21 release include: * Add `v1beta2` version for all resources. `v1beta2` removes all deprecated fields. * Add annotations that enable the operator to restart Kafka Connect connectors or tasks. The annotations can be applied to the KafkaConnector and the KafkaMirrorMaker2 custom resources. * Add additional configuration options for the Kaniko executor used by the Kafka Connect Build on Kubernetes * Add support for JMX options configuration of all Kafka Connect (KC, KC2SI, MM2) * Update Strimzi Kafka OAuth to version 0.7 and add support for new features: * OAuth authentication over SASL PLAIN mechanism * Checking token audience * Validating tokens using JSONPath filter queries to perform custom checks * Fix Cruise Control crash loop when updating container configurations * Configure external logging `ConfigMap` name and key. * Add support for configuring labels and annotations in ClusterRoleBindings created as part of Kafka and Kafka Connect clusters * Add support for Ingress v1 in Kubernetes 1.19 and newer * Add support for Kafka 2.6.1 * List topics used by a Kafka Connect connector in the `.status` section of the `KafkaConnector` custom resource * Bump Cruise Control to v2.5.37 for Kafka 2.7 support. Note this new version of Cruise Control uses `Log4j 2` and is supported by dynamic logging configuration (where logging properties are defined in a ConfigMap). However, existing `Log4j` configurations must be updated to `Log4j 2` configurations. * Support pausing reconciliation of CR with annotation `strimzi.io/pause-reconciliation` There are also several deprecations and removals and one known issue. For more details and installation files, go to https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.22.0 Thanks to everyone who contributed to this release! |
|
|
|
RC1 of Strimzi Operators 0.22.0
Jakub Scholz
Release Candidate 1 of Strimzi Operators 0.22.0 is now available for testing with a lot of changes and improvements. This release introduces new API version `v1beta2` to all Strimzi custom resources. This is a preparation for migration to `apiextensions/v1` which is needed because Kubernetes 1.22 will remove support for `apiextensions/v1beta1`. Migration to `v1beta2` needs to be completed for all Strimzi CRDs and CRs after the upgrade to 0.22 is done and before upgrading to Strimzi 0.23 which will support only Strimzi `v1beta2` APIs and `apiextensions/v1` CRDs. For more details about the CRD upgrades, see the documentation: https://strimzi.io/docs/operators/master/deploying.html#assembly-upgrade-resources-str The main changes since the 0.21 release include: * Add `v1beta2` version for all resources. `v1beta2` removes all deprecated fields. * Add annotations that enable the operator to restart Kafka Connect connectors or tasks. The annotations can be applied to the KafkaConnector and the KafkaMirrorMaker2 custom resources. * Add additional configuration options for the Kaniko executor used by the Kafka Connect Build on Kubernetes * Add support for JMX options configuration of all Kafka Connect (KC, KC2SI, MM2) * Update Strimzi Kafka OAuth to version 0.7 and add support for new features: * OAuth authentication over SASL PLAIN mechanism * Checking token audience * Validating tokens using JSONPath filter queries to perform custom checks * Fix Cruise Control crash loop when updating container configurations * Configure external logging `ConfigMap` name and key. * Add support for configuring labels and annotations in ClusterRoleBindings created as part of Kafka and Kafka Connect clusters * Add support for Ingress v1 in Kubernetes 1.19 and newer * Add support for Kafka 2.6.1 * List topics used by a Kafka Connect connector in the `.status` section of the `KafkaConnector` custom resource * Bump Cruise Control to v2.5.37 for Kafka 2.7 support. Note this new version of Cruise Control uses `Log4j 2` and is supported by dynamic logging configuration (where logging properties are defined in a ConfigMap). However, existing `Log4j` configurations must be updated to `Log4j 2` configurations. * Support pausing reconciliation of CR with annotation `strimzi.io/pause-reconciliation` There are also several deprecations and removals. For more details and installation files, go to https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.22.0-rc1 Any feedback can be provided on the Strimzi mailing list, on the #strimzi Slack channel on CNCF Slack or as a GitHub issue. Thanks & Regards Jakub & Strimzi team |
|
|
|
[ANNOUNCE] [RELEASE] Strimzi Kafka OAuth library 0.7.1 released
Jakub Scholz
Hi, Version 0.7.1 of the Strimzi Kafka OAuth library is now available: https://github.com/strimzi/strimzi-kafka-oauth/releases/tag/0.7.1. The main changes since 0.7.0 are two bugfixes: * Fixed OAuth over PLAIN intermittent failures (#95) * Fix NPE in Keycloak Authorizer (#97) For more details about the new features see the RELEASE_NOTES and the README files. All changes can be found under the 0.7.1 milestone. Thanks to everyone who contributed to this release! Thanks & Regards Strimzi team |
|
|
|
[ANNOUNCE] [RELEASE] Strimzi Kafka OAuth library 0.7.0 released
Jakub Scholz
Hi, Version 0.7.0 of the Strimzi Kafka OAuth library is now available: https://github.com/strimzi/strimzi-kafka-oauth/releases/tag/0.7.0. The main changes since 0.6.x are: * OAuth authentication over SASL PLAIN * Checking `audience` of the JWT token in the server part of the OAuth library * Custom claim checking For more details about the new features see the RELEASE_NOTES and the README files. All changes can be found under the 0.7.0 milestone. Thanks to everyone who contributed to this release! Thanks & Regards Strimzi team |
|
|
|
RC2 of Strimzi Kafka OAuth library 0.7.0
Jakub Scholz
Hi, Release Candidate 2 of the 0.7.0 version of the Strimzi Kafka OAuth library is now available for testing: https://github.com/strimzi/strimzi-kafka-oauth/releases/tag/0.7.0-rc2. Compared to RC1, it adds the custom claim checking feature and test improvements. The main changes since 0.6.x are: * OAuth authentication over SASL PLAIN * Checking `audience` of the JWT token in the server part of the OAuth library * Custom claim checking To test it, you can use the staging Maven repository: <repositories> <repository> <id>staging</id> <url>https://oss.sonatype.org/content/repositories/iostrimzi-1090</url> </repository> </repositories> For more details about the new features see the RELEASE_NOTES and the README files. All changes can be found under the 0.7.0 milestone. Any feedback can be provided on the mailing list, on Slack or as a GitHub issue. Thanks & Regards Jakub |
|
|
|
[ANNOUNCE] [RELEASE] Strimzi Kafka Operators 0.21.1 released
Jakub Scholz
Shortly after releasing 0.21.0 we discovered two bugs affecting it. That is why we now released Strimzi Operators 0.21.1 with the following bug-fixes: * Fix broken links in the OAuth documentation (#4265) * Fix the network-policies handling when metrics config from CM is used (#4261) For more details and installation files, go to https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.21.1 Thanks & Regards Strimzi team |
|
|
|
Re: [ANNOUNCE] [RELEASE] Strimzi Kafka Operators 0.21.0 released
Jakub Scholz
We found a bug in the new feature for configuring metrics from ConfigMap instead of directly in the custom resource. When used, it in some cases does not properly configure the network policies for the port 9404 used for the metrics. As a workaround, either keep using the old configuration or create manually an additional network policy with your custom name for the port 9404. This bug will be fixed in 0.21.1. Thanks & Regards Jakub
|
|
|
|
[ANNOUNCE] [RELEASE] Strimzi Kafka Operators 0.21.0 released
Jakub Scholz
Strimzi Operators 0.21.0 is now available with a lot of important changes and improvements. The main changes since the 0.20.0 release include: * Add support for Kafka 2.7.0 * Add support for declarative management of connector plugins in Kafka Connect CR * Add `inter.broker.protocol.version` to the default configuration in example YAMLs * Add support for `secretPrefix` property for User Operator to prefix all secret names created from KafkaUser resource. * Allow configuring labels and annotations for Cluster CA certificate secrets * Add the JAAS configuration string in the sasl.jaas.config property to the generated secrets for KafkaUser with SCRAM-SHA-512 authentication. * Strimzi `test-container` has been renamed to `strimzi-test-container` to make the name more clear * Updated the CPU usage metric in the Kafka, ZooKeeper and Cruise Control dashboards to include the CPU kernel time (other than the current user time) * Allow disabling ownerReference on CA secrets * Make it possible to run Strimzi operators and operands with read-only root filesystem * Move from Docker Hub to Quay.io as our container registry * Add possibility to configure DeploymentStrategy for Kafka Connect, Kafka Mirror Maker (1 and 2), and Kafka Bridge * Support passing metrics configuration as an external ConfigMap * Enable CORS configuration for Cruise Control * Add support for rolling individual Kafka or ZooKeeper pods through the Cluster Operator using an annotation * Add support for Topology Spread Constraints in Pod templates * Make Kafka `cluster-id` (KIP-78) available on Kafka CRD status There are also some deprecations you should be aware of: * The `metrics` field in the Strimzi custom resources has been deprecated and will be removed in the future. For configuring metrics, use the new `metricsConfig` field and pass the configuration via ConfigMap. Since 0.21.0 Strimzi supports Kubernetes version 1.16 and newer. For more details and installation files, go to https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.21.0 Many thanks to everyone who contributed to this release! Thanks & Regards Strimzi team |
|
|
|
RC1 of Strimzi Operators 0.21.0
Jakub Scholz
Release Candidate 1 of Strimzi Operators 0.21.0 is now available for testing with a lot of changes and improvements. The main changes since the 0.20.0 release include: * Add support for declarative management of connector plugins in Kafka Connect CR * Add `inter.broker.protocol.version` to the default configuration in example YAMLs * Add support for `secretPrefix` property for User Operator to prefix all secret names created from KafkaUser resource. * Allow configuring labels and annotations for Cluster CA certificate secrets * Add the JAAS configuration string in the sasl.jaas.config property to the generated secrets for KafkaUser with SCRAM-SHA-512 authentication. * Strimzi `test-container` has been renamed to `strimzi-test-container` to make the name more clear * Updated the CPU usage metric in the Kafka, ZooKeeper and Cruise Control dashboards to include the CPU kernel time (other than the current user time) * Allow disabling ownerReference on CA secrets * Make it possible to run Strimzi operators and operands with read-only root filesystem * Move from Docker Hub to Quay.io as our container registry * Add possibility to configure DeploymentStrategy for Kafka Connect, Kafka Mirror Maker (1 and 2), and Kafka Bridge * Support passing metrics configuration as an external ConfigMap * Enable CORS configuration for Cruise Control * Add support for rolling individual Kafka or ZooKeeper pods through the Cluster Operator using an annotation * Add support for Topology Spread Constraints in Pod templates * Make Kafka `cluster-id` (KIP-78) available on Kafka CRD status There are also some deprecations and removals which you should be aware of: * The `metrics` field in the Strimzi custom resources has been deprecated and will be removed in the future. For configuring metrics, use the new `metricsConfig` field and pass the configuration via ConfigMap. This version of Strimzi supports Kubernetes 1.16 and higher. For more details and installation files, go to https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.21.0-rc1 Any feedback can be provided on the Strimzi mailing list, on the #strimzi Slack channel on CNCF Slack or as a GitHub issue. Thanks & Regards Jakub & Strimzi team |
|
|
|
[ANNOUNCE] [RELEASE] Strimzi Kafka Operators 0.20.1 released
Jakub Scholz
Strimzi Operators 0.20.1 is now available with several bug-fixes: * Silent error from missing CRB RBAC in Kafka Connect when not needed (#4019) * Fine-tune the Kafka Exporter health checks (#3885) * Correct connect configuration comparison (#3987) * Avoid changing custom resource status because of HashSet ordering (#4069) * Fix the client rack-awareness in Kafka Connect (#3903) * Connect default logging not expanded (#4057) * Topic operator improvements (#3982) * Do not use ownerReference in UO and TO bindings into a different namespace (#4080) * Remove owner references from ClusterRoleBindings (#4077) * Topic Operator metrics (#3883) For more details and installation files, go to https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.20.1 Many thanks to everyone who contributed to this release! Thanks & Regards Strimzi team |
|
|
|
RC1 of Strimzi Operators 0.20.1
Jakub Scholz
Release Candidate 1 of Strimzi Operators 0.20.1 is now available for testing with several bug-fixes: * Silent error from missing CRB RBAC in Kafka Connect when not needed (#4019) * Fine-tune the Kafka Exporter health checks (#3885) * Correct connect configuration comparison (#3987) * Avoid changing custom resource status because of HashSet ordering (#4069) * Fix the client rack-awareness in Kafka Connect (#3903) * Connect default logging not expanded (#4057) * Topic operator improvements (#3982) * Do not use ownerReference in UO and TO bindings into a different namespace (#4080) * Remove owner references from ClusterRoleBindings (#4077) * Topic Operator metrics (#3883) For more details and installation files, go to https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.20.1-rc1 Any feedback can be provided on the Strimzi mailing list, on the #strimzi Slack channel on CNCF Slack or as a GitHub issue. Thanks & Regards Jakub |
|
|
|
RC1 of Strimzi Kafka OAuth library 0.7.0
Jakub Scholz
Hi, Release Candidate 1 of the 0.7.0 version of the Strimzi Kafka OAuth library is now available for testing: https://github.com/strimzi/strimzi-kafka-oauth/releases/tag/0.7.0-rc1. The main changes are: * OAuth authentication over SASL PLAIN * Checking `audience` of the JWT token in the server part of the OAuth library To test it, you can use the staging Maven repository: <repositories> <repository> <id>staging</id> <url>https://oss.sonatype.org/content/repositories/iostrimzi-1084</url> </repository> </repositories> Any feedback can be provided on the mailing list, on Slack or as a GitHub issue. Thanks & Regards Jakub |
|
|
|
[ANNOUNCE] [RELEASE] Strimzi Kafka Operators 0.20.0 released
Jakub Scholz
Strimzi Operators 0.20.0 is now available with a lot of important changes and improvements. The main changes since the 0.19.0 release include: * Add support for Kafka 2.5.1 and 2.6.0. Remove support for 2.4.0 and 2.4.1 * Make `listeners` configurable as an array and add support for more different listeners in single cluster * Remove TLS sidecars from Kafka pods => Kafka now uses native TLS to connect to ZooKeeper * Updated to Cruise Control 2.5.11, which adds Kafka 2.6.0 support and fixes a previous issue with CPU utilization statistics for containers. As a result, the CPUCapacityGoal has now been enabled. * Configure Cluster Operator logging using ConfigMap instead of environment variable and support dynamic changes * Switch to use the `AclAuthorizer` class for the `simple` Kafka authorization type. `AclAuthorizer` contains new features such as the ability to control the amount of authorization logs in the broker logs. * Support dynamic logging configuration of Kafka, Kafka Connect, Kafka Connect S2I and Kafka MirrorMaker2 * Add support for `client.rack` property for Kafka Connect to use `fetch from closest replica` feature. * Add support for configuring `hostAliases` in Pod templates * Cruise Control metrics integration including a new Grafana dashboard * Improved operators Grafana dashboard * Improvements to OAuth authentication and Keycloak authorization There are also some deprecations and removals which you should be aware of: * Deprecation of the old listener configuration * Removal of monitoring port on Kafka and ZooKeeper related services * Deprecation of Kafka TLS sidecar configuration 0.20.0 will also be the last version of Strimzi that will support Kubernetes 1.11-1.15. Future versions will support Kubernetes 1.16+. For more details and installation files, go to https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.20.0 Many thanks to everyone who contributed to this release! Thanks & Regards Strimzi team |
|
|
|
RC3 of Strimzi Operators 0.20.0
Jakub Scholz
Release Candidate 3 of Strimzi Operators 0.20.0 is now available for testing. Compared to the Release Candidate 2, there are several bugfixes and improvements: * Fix rolling updates when all Kafka pods are Pending (#3848) * Bump OPA Authorizer version to fix CVE in its dependencies (#3853) * Fix rebalance examples (#3838) * Fix cAdvisor metric labels (#3840) * Several docs and system test fixes and improvements Note: 0.20.0 will be the last version of Strimzi that will support Kubernetes 1.11-1.15. Future versions will support Kubernetes 1.16+. For more details and installation files, go to https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.20.0-rc3 Any feedback can be provided on the Strimzi mailing list, on the #strimzi Slack channel on CNCF Slack or as a GitHub issue. Thanks & Regards Jakub |
|
|
|
RC2 of Strimzi Operators 0.20.0
Jakub Scholz
Release Candidate 2 of Strimzi Operators 0.20.0 is now available for testing. Compared to the Release Candidate 1, there are several bugfixes and improvements: * Fix secrets with `.` in name (#3820) * Fix leak of HTTPClients in Connector Operator (#3827) * Fix handling of custom and OAuth certificates (#3825) * Upgrade JMX Exporter to 0.14.0 * Several docs and system test fixes and improvements Note: 0.20.0 will be the last version of Strimzi that will support Kubernetes 1.11-1.15. Future versions will support Kubernetes 1.16+. For more details and installation files, go to https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.20.0-rc2 Any feedback can be provided on the Strimzi mailing list, on the #strimzi Slack channel on CNCF Slack or as a GitHub issue. Thanks & Regards Jakub |
|
|
|
RC1 of Strimzi Operators 0.20.0
Jakub Scholz
Hi, Release Candidate 1 of Strimzi Operators 0.20.0 is now available for testing with a lot of changes and improvements. The main changes since the 0.19.0 release include: * Add support for Kafka 2.5.1 and 2.6.0. Remove support for 2.4.0 and 2.4.1 * Make `listeners` configurable as an array and add support for more different listeners in single cluster * Remove TLS sidecars from Kafka pods => Kafka now uses native TLS to connect to ZooKeeper * Updated to Cruise Control 2.5.11, which adds Kafka 2.6.0 support and fixes a previous issue with CPU utilization statistics for containers. As a result, the CPUCapacityGoal has now been enabled. * Configure Cluster Operator logging using ConfigMap instead of environment variable and support dynamic changes * Switch to use the `AclAuthorizer` class for the `simple` Kafka authorization type. `AclAuthorizer` contains new features such as the ability to control the amount of authorization logs in the broker logs. * Support dynamic logging configuration of Kafka, Kafka Connect, Kafka Connect S2I and Kafka MirrorMaker2 * Add support for `client.rack` property for Kafka Connect to use `fetch from closest replica` feature. * Add support for configuring `hostAliases` in Pod templates * Cruise Control metrics integration including a new Grafana dashboard * Improved operators Grafana dashboard There are also some deprecations and removals which you should be aware of: * Deprecation of the old listener configuration * Removal of monitoring port on Kafka and ZooKeeper related services * Deprecation of Kafka TLS sidecar configuration 0.20.0 will also be the last version of Strimzi that will support Kubernetes 1.11-1.15. Future versions will support Kubernetes 1.16+. For more details and installation files, go to https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.20.0-rc1 Any feedback can be provided on the Strimzi mailing list, on the #strimzi Slack channel on CNCF Slack or as a GitHub issue. Thanks & Regards Jakub |
|
|
