|
RCs for Strimzi 0.14.0
Jakub Scholz
The Release Candidate 1 for Strimzi Kafka Operators and Release Candidate 2 for Strimzi Bridge are now available. There is a lot of new features, so if you are interested in any of them feel free to test them and give us feedback. The main changes in the operators include: * Add support for configuring Ingress class (#1716) * Add support for setting custom environment variables in all containers * Add liveness and readiness checks to Mirror Maker * Allow configuring loadBalancerIP for LoadBalancer type services * Allow setting labels and annotations for Persistent Volume Claims * Add support for Jaeger tracing in Kafka Mirror Maker and Kafka Connect * Add support for deploying [Kafka Exporter](https://github.com/danielqsj/kafka_exporter) * Add initial support for OAuth authentication For more details and the upgrade procedure, go to: https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.14.0-rc1 The main changes in the Bridge include: * Changed data types and enforced OpenAPI validation for `consumer.request.timeout.ms`, `enable.auto.commit`and `fetch.min.bytes`parameters on consumer creation. This is a breaking change. * Added HTTP GET method on `/consumers/{groupid}/instances/{name}/subscription` endpoint for getting subscribed topics and related assigned partitions. * Added automatic deletion of stale consumer after a configurable timeout if the HTTP DELETE is not called and the consumer is not used for long time. * Various bug fixes. For more details, go to: https://github.com/strimzi/strimzi-kafka-bridge/releases/tag/0.14.0-rc2 Thanks & Regards Jakub |
|
|
|
RC2 of Strimzi Kafka Operators 0.14.0
Jakub Scholz
We found a few bugs in the 0.14.0 RC1 release of the Strimzi operators. I therefore prepared RC2. The main changes since RC1 include: * Manage network policies between Kafka Exporter and Kafka brokers. * Documentation improvements * CI improvements * System test improvements For more details and the upgrade procedure, go to: https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.14.0-rc2 Thanks & Regards Jakub |
|
|
|
0.14.0 RC3 of Strimzi Kafka Operators and Strimzi Bridge
Jakub Scholz
We prepared RC3 of Strimzi Kafka Operators and Strimzi Bridge. The main changes since RC2 in the operators include: * Updated dependencies due to CVEs (jackson-databind library) * Properly passing JWKS refresh and expiration intervals in OAuth configuration * Respecting the HTTP port configuration in Strimzi bridge * Using the right default container images for TLS sidecars * Other improvements to Maven build, Docs and system tests The main changes since RC2 in the bridge include: * Added logging of the bridge version on startup * Minor bug fixes For more details and the upgrade procedure, go to: https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.14.0-rc3 and https://github.com/strimzi/strimzi-kafka-bridge/releases/tag/0.14.0-rc3 Unless we found some more serious issues, we will release 0.14.0 during the weekend. Jakub |
|
|
|
Strimzi 0.14.0 released!
Jakub Scholz
Version 0.14.0 of Strimzi Kafka Operators and Strimzi Bridge has been released. Thanks to everyone involved in these releases. The main changes in the operators include: * Add support for configuring Ingress class * Add support for setting custom environment variables in all containers * Add liveness and readiness checks to Mirror Maker * Allow configuring loadBalancerIP for LoadBalancer type services * Allow setting labels and annotations for Persistent Volume Claims * Add support for Jaeger tracing in Kafka Mirror Maker and Kafka Connect * Add support for deploying Kafka Exporter * Add initial support for OAuth authentication For more details and the upgrade procedure, go to: https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.14.0 The main changes in the Bridge include: * Changed data types and enforced OpenAPI validation for `consumer.request.timeout.ms`, `enable.auto.commit`and `fetch.min.bytes`parameters on consumer creation. This is a breaking change. * Added HTTP GET method on `/consumers/{groupid}/instances/{name}/subscription` endpoint for getting subscribed topics and related assigned partitions. * Added automatic deletion of stale consumer after a configurable timeout if the HTTP DELETE is not called and the consumer is not used for a long time. * Various bug fixes. For more details, go to: https://github.com/strimzi/strimzi-kafka-bridge/releases/tag/0.14.0 Thanks & Regards Jakub |
|
|
|
RC1 of Strimzi Kafka OAuth library 0.2.0
Jakub Scholz
Hi, Release Candidate 1 of the 0.2.0 version of the Strimzi KAfka OAuth library is now available for testing: https://github.com/strimzi/strimzi-kafka-oauth/releases/tag/0.2.0-rc1. The main improvement is improved support for other OAuth servers than just Keycloak. To test it, you can use the staging Maven repository: <repositories> <repository> <id>staging</id> <url>https://oss.sonatype.org/content/repositories/iostrimzi-1046/</url> </repository> </repositories> Any feedback can be provided here or as a GitHub issue. Thanks & Regards Jakub |
|
|
|
First RCs for Strimzi 0.15.0
Jakub Scholz
The Release Candidate 1 for Strimzi Kafka Operators and for Strimzi Bridge are now available. There is a lot of new features (especially for the operators), so if you are interested in any of them feel free to test them and give us feedback. The main changes in the operators include: * Drop support for Kafka 2.1.0, 2.1.1, and 2.2.0 * Add support for Kafka 2.3.1 * Improved Kafka rolling update * Improved Kafka Exporter Grafana dashboard * Add sizeLimit option to ephemeral storage * Add `schedulerName` to `podTemplate` * Allow overriding the auto-detected Kubernetes version * Garbage Collection (GC) logging disabled by default * Providing PKCS12 truststore and password in the cluster and clients CA certificates Secrets * Providing PKCS12 keystore and password in the TLS based KafkaUser related Secret For more details and the upgrade procedure, go to: https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.15.0-rc1 The main changes in the Bridge include: * Added support for Jaeger tracing * Various bug fixes. For more details, go to: https://github.com/strimzi/strimzi-kafka-bridge/releases/tag/0.15.0-rc1 Thanks & Regards Jakub |
|
|
|
[ANNOUNCE] Strimzi Kafka OAuth library 0.2.0 release
Jakub Scholz
Hi, The 0.2.0 version of the Strimzi Kafka OAuth library for using OAuth authentication in Kafka clients and brokers has been released and should be available in the Maven repositories. The main improvement is improved support for other OAuth servers than just Keycloak. For more information, see the release page on GitHub: https://github.com/strimzi/strimzi-kafka-oauth/releases/tag/0.2.0. Thanks to everyone who contributed to this release. Thanks & Regards Jakub |
|
|
|
Re: [cncf-strimzi-dev] First RCs for Strimzi 0.15.0
Tom Bentley
+1 I did some testing on minikube of the features mentioned in the release nodes and everything seemed to work as expected. Thanks Jakub!
|
|
|
|
[ANNOUNCE] Strimzi 0.15.0 released!
Jakub Scholz
Version 0.15.0 of Strimzi Kafka Operators and Strimzi Bridge has been released. The main changes in the operators include: * Drop support for Kafka 2.1.0, 2.1.1, and 2.2.0 * Add support for Kafka 2.3.1 * Improved Kafka rolling update * Improved Kafka Exporter Grafana dashboard * Add sizeLimit option to ephemeral storage * Add `schedulerName` to `podTemplate` * Allow overriding the auto-detected Kubernetes version * Garbage Collection (GC) logging disabled by default * Providing PKCS12 truststore and password in the cluster and clients CA certificates Secrets * Providing PKCS12 keystore and password in the TLS based KafkaUser related Secret For more details and the upgrade procedure, go to: https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.15.0 The main changes in the Bridge include: * Added support for Jaeger tracing * Various bug fixes. For more details, go to: https://github.com/strimzi/strimzi-kafka-bridge/releases/tag/0.15.0 Thanks to everyone who contributed to these releases. Jakub |
|
|
|
strimzi for debezium with a managed fafka (eventhub kafka facade)
alonisser@...
I want to use strimzi to deploy debezium with kafka connect and eventhub kafka facade in my aks cluster, All the tutorial I've encountered are about using strimzi to create a kafka cluster (which I don't need)
My question: 1. What components I need to deploy for debezium with a managed kafka ? (users? topics? or does debezium handle that) 2. How can I control with strimzi what components to deploy 3. How can I provide the kafka connection configuration to the components that need it? Thanks for the help! |
|
|
|
Re: strimzi for debezium with a managed fafka (eventhub kafka facade)
Jakub Scholz
Hi, The Kafka Connect deployment with Strimzi can work independently on the KAfka broker and there are users using Strimzi just for Connect for example with Amazon MSK etc. You would still need to install the Strimzi Cluster Operator as described in all the guides - it is the same operator which is managing Kafka cluster which also manages Kafka Connect. Once you have the cluster operator running, you can just go and deploy KAfka Connect - check this chapter for more details to get you started: https://strimzi.io/docs/latest/full.html#kafka-connect-str (eventually this for a more detailed description of the different options: https://strimzi.io/docs/latest/full.html#assembly-deployment-configuration-kafka-connect-str). In the KafkaConnect custom resource, you specify the bootstrap address directly as hostname:port - so you can just point it to your broker (plus you will probably need to configure some auth etc.) I'm not sure if anyone ever tried it with Azure Event Hub, but I guess it should work. Thanks & Regards Jakub On Thu, Jan 2, 2020 at 1:18 PM <alonisser@...> wrote: I want to use strimzi to deploy debezium with kafka connect and eventhub kafka facade in my aks cluster, All the tutorial I've encountered are about using strimzi to create a kafka cluster (which I don't need) |
|
|
|
Release Candidate 1 of Strimzi Kafka Operators 0.16.0
Jakub Scholz
The Release Candidate 1 for Strimzi Kafka Operators 0.16.0 is now available. There is a lot of new features, so if you are interested in any of them please test them and give us feedback. The main changes since 0.15.0 include: * Add support for Kafka 2.4.0 and upgrade from Zookeeper 3.4.x to 3.5.x * Drop support for Kafka 2.2.1 and 2.3.0 * Add KafkaConnector resource and connector operator * Let users choose which node address will be used as advertised host (ExternalDNS, ExternalIP, InternalDNS, InternalIP or Hostname) * Add support for tini * When not explicitly configured by the user in jvmOptions, -Xmx option is calculated from memory requests rather than from memory limits * Expose JMX port on Kafka brokers via an internal service * Add support for externalTrafficPolicy and loadBalancerSourceRanges properties on loadbalancer and nodeport type services * Support for user quotas * Possibility to specify custom server certificates to TLS based listeners * Add support for Istio protocol selection in service port names * Note: Strimzi is essentially adding a tcp- prefix to the port names in Kafka services and headless services. (e.g clientstls -> tcp-clientstls) For more details and the upgrade procedure, go to: https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.16.0-rc1 Thanks & Regards Jakub |
|
|
|
Release Candidate 2 of Strimzi Kafka Operators 0.16.0
Jakub Scholz
We found some issues in the RC1, so we prepared a new release candidate for the Strimzi 0.16.0 release. The main changes since RC1 include: * Fix to user quotas deletion * NPE in Kafka Roller * Improved documentation and system tests * and several other minor bugfixes For more details and the upgrade procedure, go to: https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.16.0-rc2 Thanks & Regards Jakub ---------- Forwarded message --------- From: Jakub Scholz <jakub@...> Date: Fri, Jan 10, 2020 at 2:31 AM Subject: Release Candidate 1 of Strimzi Kafka Operators 0.16.0 To: <cncf-strimzi-dev@...>, <cncf-strimzi-users@...> The Release Candidate 1 for Strimzi Kafka Operators 0.16.0 is now available. There is a lot of new features, so if you are interested in any of them please test them and give us feedback. The main changes since 0.15.0 include: * Add support for Kafka 2.4.0 and upgrade from Zookeeper 3.4.x to 3.5.x * Drop support for Kafka 2.2.1 and 2.3.0 * Add KafkaConnector resource and connector operator * Let users choose which node address will be used as advertised host (ExternalDNS, ExternalIP, InternalDNS, InternalIP or Hostname) * Add support for tini * When not explicitly configured by the user in jvmOptions, -Xmx option is calculated from memory requests rather than from memory limits * Expose JMX port on Kafka brokers via an internal service * Add support for externalTrafficPolicy and loadBalancerSourceRanges properties on loadbalancer and nodeport type services * Support for user quotas * Possibility to specify custom server certificates to TLS based listeners * Add support for Istio protocol selection in service port names * Note: Strimzi is essentially adding a tcp- prefix to the port names in Kafka services and headless services. (e.g clientstls -> tcp-clientstls) For more details and the upgrade procedure, go to: https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.16.0-rc1 Thanks & Regards Jakub |
|
|
|
KafkaConnect resource - how to use SASL_SSL
alonisser@...
For connection for my kafka instance (by eventhub kafka facade) I need the following properties
security.protocol=SASL_SSL producer.sasl.mechanism=SASL_SSL consumer.sasl.mechanism=SASL_SSL I've verified that with them this does work as expected But I can't find how to configure the KafkaConnect resource to use them I can't use "config" directly because those properties are limited to strimzi managment only and are disregarded I've followed kafka_connect_config_generator.sh generator (that generates the /tmp/ properties file the connect actually runs and found those lines if [ "$KAFKA_CONNECT_TLS" = "true" ]; then
SECURITY_PROTOCOL="SSL"
if [ -n "$KAFKA_CONNECT_SASL_MECHANISM" ]; then
if [ "$SECURITY_PROTOCOL" = "SSL" ]; then
SECURITY_PROTOCOL="SASL_SSL"
else
SECURITY_PROTOCOL="SASL_PLAINTEXT"
fi
And checking the actually generated file I see I have SASL_PLAINTEXT So the question is how do I setup tls to true, without needing to add the certificates/secrets etc.. (because the properties file that does work, doesn't actually need them) using 16.0 rc1 |
|
|
|
Re: KafkaConnect resource - how to use SASL_SSL
alonisser@...
Turns out that adding
ssl :{} in the spec. does the trick :) |
|
|
|
[ANNOUNCE] Strimzi Kafka operator 0.16.0
Jakub Scholz
Strimzi Kafka Operators 0.16.0 is now available with a lot of great new features. The main changes since 0.15.0 include: * Add support for Kafka 2.4.0 and upgrade from Zookeeper 3.4.x to 3.5.x * Drop support for Kafka 2.2.1 and 2.3.0 * Add KafkaConnector resource and connector operator * Let users choose which node address will be used as advertised host (ExternalDNS, ExternalIP, InternalDNS, InternalIP or Hostname) * Add support for tini * When not explicitly configured by the user in jvmOptions, -Xmx option is calculated from memory requests rather than from memory limits * Expose JMX port on Kafka brokers via an internal service * Add support for externalTrafficPolicy and loadBalancerSourceRanges properties on loadbalancer and nodeport type services * Support for user quotas * Possibility to specify custom server certificates to TLS based listeners * Add support for Istio protocol selection in service port names * Note: Strimzi is essentially adding a tcp- prefix to the port names in Kafka services and headless services. (e.g clientstls -> tcp-clientstls) For more details go to: https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.16.0 Thanks to everyone who contributed to these releases. Jakub |
|
|
|
[ANNOUNCE] Strimzi Kafka operator 0.16.1
Paolo Patierno
Strimzi Kafka Operators 0.16.1 patch release is now available. The main changes since 0.16.0 include:
For more details go to: https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.16.1
Thanks to everyone who contributed to these releases. Paolo |
|
|
|
RC1 of Strimzi Kafka OAuth library 0.3.0
Jakub Scholz
Hi, Release Candidate 1 of the 0.3.0 version of the Strimzi Kafka OAuth library is now available for testing: https://github.com/strimzi/strimzi-kafka-oauth/releases/tag/0.3.0-rc1. The main improvements are support for ECDSA signatures in JWT tokens and support for Keycloak Authorization Services. To test it, you can use the staging Maven repository: <repositories> <repository> <id>staging</id> <url>https://oss.sonatype.org/content/repositories/iostrimzi-1054/</url> </repository> </repositories> Any feedback can be provided here or as a GitHub issue. Thanks & Regards Jakub |
|
|
|
[ANNOUNCE] Strimzi Kafka operator 0.16.2
Jakub Scholz
Strimzi Kafka Operators 0.16.2 is now available with two bug fixes: * Fix bug in upgrade when the Kafka.spec.kafka.version field is not specified * Fix network policies for port 3.11 on older Kubernetes versions For more details go to: https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.16.2 Thanks to everyone who contributed to these releases. Jakub |
|
|
|
[ANNOUNCE] Strimzi Kafka OAuth library 0.3.0 release
Jakub Scholz
The 0.3.0 version of the Strimzi Kafka OAuth library for using OAuth authentication in Kafka clients and brokers has been released and should be available in the Maven repositories. The main improvements are: * The main improvements are support for ECDSA signatures in JWT tokens * Support for Keycloak Authorization Services. For more information, see the release page on GitHub: https://github.com/strimzi/strimzi-kafka-oauth/releases/tag/0.3.0 Thanks to everyone who contributed to this release. Thanks & Regards Jakub |
|
|