Date   

RC2 of Strimzi Kafka Operators 0.17.0 is now available

Jakub Scholz
 

Release Candidate 2 of Strimzi Kafka Operators 0.17.0 is now available. It took us more than two weeks since the RC1, but we spent them with a lot of testing and improving the code to make sure the quality of the 0.17 release is good. The main changes since RC1 include:
* Support for pausing / resuming MM2 connectors
* Fix bug in Kafka rack awareness configuration
* Add network policies for Kafka connect when using the connector operator
* Fix rolling update restart when configuration changes
* Fix ZooKeeper scale-up bug
* Validate the replication factor in relation to number of Kafka replicas
* Various bug-fixes
* Dependency upgrades to avoid CVEs
* Improved system tests and documentation

For more details about the release candidate and the upgrade procedure, please go to https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.17.0-rc2

Thanks & Regards
Jakub


Re: Does anyone here has an experience with running schema manager (for avro) with strimzi kafka ?

Matthew Isaacs
 

I used the open source helm chart from Confluent: https://github.com/confluentinc/cp-helm-charts/tree/v5.4.0/charts/cp-schema-registry

The heap settings and resource limits were not sufficient for the 5.4.0 release (I need to send in a patch for this and other issues).

## Schema registry JVM Heap Option
heapOptions: "-Xms1000M -Xmx1000M"
resources:
  limits:
   memory: 2Gi
Otherwise, don't forget to specify the your Kafka bootstrap endpoint:
kafka:
  bootstrapServers: "Xx-Kafka-bootstrap....:9092"

On Wed, Feb 12, 2020 at 1:51 PM alonn <alonisser@...> wrote:

>
> Thanks, Yes, I'm trying with multiple instances. I can do with one, probably if that works
> Can you please provide a sample deployment.yaml ? so I can see what I'm missing? 
> Tech blog:https://medium.com/@alonisser
> Personal Blog:degeladom.wordpress.com
> Tel:972-54-6734469
> Email: alonisser@...
>
>
>
> On Wed, Feb 12, 2020 at 7:51 PM Matthew Isaacs <matthew.isaacs@...> wrote:
>>
>> I'm running the Schema Registry service along side a Strimzi deployed Kafka. It seems to be working fine for me. Are you trying to run multiple instances of the the schema registry? I'm afraid I don't fully understand what problem you're having.


Does anyone here has an experience with running schema manager (for avro) with strimzi kafka ?

alonisser@...
 

I've setup the most basic kafka persistent setup (see the crd below)
But can't seem to understand how to run schema registy against it (with kafka based leader selection)
I see kafka logs with endless generations like this one
Preparing to rebalance group schema-registry in state PreparingRebalance with old generation 24
So it does seem to actually connect , but then something goes wrong

Using bootstrap servers leading to the kafka service 9092 port
bootstrap.servers = [PLAINTEXT://my-fafka-bootstrap:9092]
 

The kafka crd

apiVersion: kafka.strimzi.io/v1beta1
kind: Kafka
metadata:
name: "internal-kafka"
spec:
kafka:
version: 2.4.0
replicas: 1
listeners:
plain: {}
tls: {}
config:
offsets.topic.replication.factor: 1
transaction.state.log.replication.factor: 1
transaction.state.log.min.isr: 1
log.message.format.version: "2.4"
storage:
type: jbod
volumes:
- id: 0
type: persistent-claim
size: 100Gi
class: managed-premium
deleteClaim: false
zookeeper:
replicas: 1
storage:
type: persistent-claim
size: 100Gi
class: managed-premium
deleteClaim: false
entityOperator:
topicOperator: {}
userOperator: {}


RC1 of Strimzi Kafka Operators 0.17.0 is now available

Jakub Scholz
 

The Release Candidate 1 of Strimzi Kafka Operators 0.17.0 is now available. It doesn't include many new features, but some of those included were in high demand. The main features are:
* Add possibility to set Java System Properties via CR yaml
* Add support for Mirror Maker 2.0
* Add Jmxtrans deployment
* Add public keys of TLS listeners to the status section of the Kafka CR

And of course a lot more smaller improvements and bugfixes. For more details about the release candidate and the upgrade procedure, please go to https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.17.0-rc1

Thanks
Jakub


[ANNOUNCE] Strimzi Kafka bridge 0.15.2

Paolo Patierno
 

Strimzi Kafka Bridge 0.15.2 is now available with the following changes:
* Fixed missing Jackson Core dependency
 
 
Thanks to everyone who contributed to these releases.
 
Paolo


[ANNOUNCE] Strimzi Kafka bridge 0.15.1

Paolo Patierno
 

Strimzi Kafka Bridge 0.15.1 is now available with the following changes:
* Bumped Apache Kafka version to 2.4.0 and other dependencies
 
 
Thanks to everyone who contributed to these releases.
 
Paolo


[ANNOUNCE] Strimzi Kafka OAuth library 0.3.0 release

Jakub Scholz
 

The 0.3.0 version of the Strimzi Kafka OAuth library for using OAuth authentication in Kafka clients and brokers has been released and should be available in the Maven repositories. The main improvements are:
* The main improvements are support for ECDSA signatures in JWT tokens
* Support for Keycloak Authorization Services. 

For more information, see the release page on GitHub: https://github.com/strimzi/strimzi-kafka-oauth/releases/tag/0.3.0 

Thanks to everyone who contributed to this release.

Thanks & Regards
Jakub


[ANNOUNCE] Strimzi Kafka operator 0.16.2

Jakub Scholz
 

Strimzi Kafka Operators 0.16.2 is now available with two bug fixes:
* Fix bug in upgrade when the Kafka.spec.kafka.version field is not specified
* Fix network policies for port 3.11 on older Kubernetes versions


Thanks to everyone who contributed to these releases.

Jakub


RC1 of Strimzi Kafka OAuth library 0.3.0

Jakub Scholz
 

Hi,

Release Candidate 1 of the 0.3.0 version of the Strimzi Kafka OAuth library is now available for testing: https://github.com/strimzi/strimzi-kafka-oauth/releases/tag/0.3.0-rc1. The main improvements are support for ECDSA signatures in JWT tokens and support for Keycloak Authorization Services. 

To test it, you can use the staging Maven repository:

  <repositories>
    <repository>
      <id>staging</id>
      <url>https://oss.sonatype.org/content/repositories/iostrimzi-1054/</url>
    </repository>
  </repositories>

Any feedback can be provided here or as a GitHub issue.

Thanks & Regards
Jakub


[ANNOUNCE] Strimzi Kafka operator 0.16.1

Paolo Patierno
 

Strimzi Kafka Operators 0.16.1 patch release is now available. The main changes since 0.16.0 include:
  • Fix connector operator when Kafka Connect runs in a different namespace
 
Thanks to everyone who contributed to these releases.

Paolo


[ANNOUNCE] Strimzi Kafka operator 0.16.0

Jakub Scholz
 

Strimzi Kafka Operators 0.16.0 is now available with a lot of great new features. The main changes since 0.15.0 include:
* Add support for Kafka 2.4.0 and upgrade from Zookeeper 3.4.x to 3.5.x
* Drop support for Kafka 2.2.1 and 2.3.0
* Add KafkaConnector resource and connector operator
* Let users choose which node address will be used as advertised host (ExternalDNS, ExternalIP, InternalDNS, InternalIP or Hostname)
* Add support for tini
* When not explicitly configured by the user in jvmOptions, -Xmx option is calculated from memory requests rather than from memory limits
* Expose JMX port on Kafka brokers via an internal service
* Add support for externalTrafficPolicy and loadBalancerSourceRanges properties on loadbalancer and nodeport type services
* Support for user quotas
* Possibility to specify custom server certificates to TLS based listeners
* Add support for Istio protocol selection in service port names
    * Note: Strimzi is essentially adding a tcp- prefix to the port names in Kafka services and headless services. (e.g clientstls -> tcp-clientstls)


Thanks to everyone who contributed to these releases.

Jakub


Re: KafkaConnect resource - how to use SASL_SSL

alonisser@...
 

Turns out that adding 
ssl :{}
in the spec. does the trick :) 


KafkaConnect resource - how to use SASL_SSL

alonisser@...
 

For connection for my kafka instance (by eventhub kafka facade) I need the following properties
security.protocol=SASL_SSL
producer.sasl.mechanism=SASL_SSL
consumer.sasl.mechanism=SASL_SSL

I've verified that with them this does work as expected 
But I can't find how to configure the KafkaConnect resource to use them
I can't use "config" directly because those properties are limited to strimzi managment only and are disregarded 
I've followed kafka_connect_config_generator.sh generator (that generates the /tmp/ properties file the connect actually runs
and found those lines
if [ "$KAFKA_CONNECT_TLS" = "true" ]; then
    SECURITY_PROTOCOL="SSL"
 
# And somewhere after that
if [ -n "$KAFKA_CONNECT_SASL_MECHANISM" ]; then
    if [ "$SECURITY_PROTOCOL" = "SSL" ]; then
        SECURITY_PROTOCOL="SASL_SSL"
    else
        SECURITY_PROTOCOL="SASL_PLAINTEXT"
    fi

And checking the actually generated file I see I have SASL_PLAINTEXT
So the question is how do I setup tls to true, without needing to add the certificates/secrets etc.. (because the properties file that does work, doesn't actually need them) 


using 16.0 rc1


Release Candidate 2 of Strimzi Kafka Operators 0.16.0

Jakub Scholz
 

We found some issues in the RC1, so we prepared a new release candidate for the Strimzi 0.16.0 release. The main changes since RC1 include:
* Fix to user quotas deletion
* NPE in Kafka Roller
* Improved documentation and system tests
* and several other minor bugfixes

For more details and the upgrade procedure, go to: https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.16.0-rc2

Thanks & Regards
Jakub

---------- Forwarded message ---------
From: Jakub Scholz <jakub@...>
Date: Fri, Jan 10, 2020 at 2:31 AM
Subject: Release Candidate 1 of Strimzi Kafka Operators 0.16.0
To: <cncf-strimzi-dev@...>, <cncf-strimzi-users@...>


The Release Candidate 1 for Strimzi Kafka Operators 0.16.0 is now available. There is a lot of new features, so if you are interested in any of them please test them and give us feedback.

The main changes since 0.15.0 include:
* Add support for Kafka 2.4.0 and upgrade from Zookeeper 3.4.x to 3.5.x
* Drop support for Kafka 2.2.1 and 2.3.0
* Add KafkaConnector resource and connector operator
* Let users choose which node address will be used as advertised host (ExternalDNS, ExternalIP, InternalDNS, InternalIP or Hostname)
* Add support for tini
* When not explicitly configured by the user in jvmOptions, -Xmx option is calculated from memory requests rather than from memory limits
* Expose JMX port on Kafka brokers via an internal service
* Add support for externalTrafficPolicy and loadBalancerSourceRanges properties on loadbalancer and nodeport type services
* Support for user quotas
* Possibility to specify custom server certificates to TLS based listeners
* Add support for Istio protocol selection in service port names
    * Note: Strimzi is essentially adding a tcp- prefix to the port names in Kafka services and headless services. (e.g clientstls -> tcp-clientstls)

For more details and the upgrade procedure, go to: https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.16.0-rc1

Thanks & Regards
Jakub


Release Candidate 1 of Strimzi Kafka Operators 0.16.0

Jakub Scholz
 

The Release Candidate 1 for Strimzi Kafka Operators 0.16.0 is now available. There is a lot of new features, so if you are interested in any of them please test them and give us feedback.

The main changes since 0.15.0 include:
* Add support for Kafka 2.4.0 and upgrade from Zookeeper 3.4.x to 3.5.x
* Drop support for Kafka 2.2.1 and 2.3.0
* Add KafkaConnector resource and connector operator
* Let users choose which node address will be used as advertised host (ExternalDNS, ExternalIP, InternalDNS, InternalIP or Hostname)
* Add support for tini
* When not explicitly configured by the user in jvmOptions, -Xmx option is calculated from memory requests rather than from memory limits
* Expose JMX port on Kafka brokers via an internal service
* Add support for externalTrafficPolicy and loadBalancerSourceRanges properties on loadbalancer and nodeport type services
* Support for user quotas
* Possibility to specify custom server certificates to TLS based listeners
* Add support for Istio protocol selection in service port names
    * Note: Strimzi is essentially adding a tcp- prefix to the port names in Kafka services and headless services. (e.g clientstls -> tcp-clientstls)

For more details and the upgrade procedure, go to: https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.16.0-rc1

Thanks & Regards
Jakub


Re: strimzi for debezium with a managed fafka (eventhub kafka facade)

Jakub Scholz
 

Hi,

The Kafka Connect deployment with Strimzi can work independently on the KAfka broker and there are users using Strimzi just for Connect for example with Amazon MSK etc. You would still need to install the Strimzi Cluster Operator as described in all the guides - it is the same operator which is managing Kafka cluster which also manages Kafka Connect. Once you have the cluster operator running, you can just go and deploy KAfka Connect - check this chapter for more details to get you started: https://strimzi.io/docs/latest/full.html#kafka-connect-str (eventually this for a more detailed description of the different options: https://strimzi.io/docs/latest/full.html#assembly-deployment-configuration-kafka-connect-str). In the KafkaConnect custom resource, you specify the bootstrap address directly as hostname:port - so you can just point it to your broker (plus you will probably need to configure some auth etc.)

I'm not sure if anyone ever tried it with Azure Event Hub, but I guess it should work.

Thanks & Regards
Jakub

On Thu, Jan 2, 2020 at 1:18 PM <alonisser@...> wrote:
I want to use strimzi to deploy debezium with kafka connect and eventhub kafka facade in my aks cluster, All the tutorial I've encountered are about using strimzi to create a kafka cluster (which I don't need)
My question:
1. What components I need to deploy for debezium with a managed kafka ? (users? topics? or does debezium handle that)  
2. How can I control with strimzi what components to deploy
3. How can I provide the kafka connection configuration to the components that need it? 

Thanks for the help!


strimzi for debezium with a managed fafka (eventhub kafka facade)

alonisser@...
 

I want to use strimzi to deploy debezium with kafka connect and eventhub kafka facade in my aks cluster, All the tutorial I've encountered are about using strimzi to create a kafka cluster (which I don't need)
My question:
1. What components I need to deploy for debezium with a managed kafka ? (users? topics? or does debezium handle that)  
2. How can I control with strimzi what components to deploy
3. How can I provide the kafka connection configuration to the components that need it? 

Thanks for the help!


[ANNOUNCE] Strimzi 0.15.0 released!

Jakub Scholz
 

Version 0.15.0 of Strimzi Kafka Operators and Strimzi Bridge has been released.

The main changes in the operators include:
* Drop support for Kafka 2.1.0, 2.1.1, and 2.2.0
* Add support for Kafka 2.3.1
* Improved Kafka rolling update
* Improved Kafka Exporter Grafana dashboard
* Add sizeLimit option to ephemeral storage
* Add `schedulerName` to `podTemplate`    
* Allow overriding the auto-detected Kubernetes version
* Garbage Collection (GC) logging disabled by default
* Providing PKCS12 truststore and password in the cluster and clients CA certificates Secrets
* Providing PKCS12 keystore and password in the TLS based KafkaUser related Secret

For more details and the upgrade procedure, go to: https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.15.0

The main changes in the Bridge include:
* Added support for Jaeger tracing
* Various bug fixes.


Thanks to everyone who contributed to these releases.

Jakub


Re: [cncf-strimzi-dev] First RCs for Strimzi 0.15.0

Tom Bentley
 

+1

I did some testing on minikube of the features mentioned in the release nodes and everything seemed to work as expected.

Thanks Jakub!



On Mon, Nov 25, 2019 at 11:13 PM Jakub Scholz via Lists.Cncf.Io <github=scholzj.com@...> wrote:
The Release Candidate 1 for Strimzi Kafka Operators and for Strimzi Bridge are now available. There is a lot of new features (especially for the operators), so if you are interested in any of them feel free to test them and give us feedback.

The main changes in the operators include:
* Drop support for Kafka 2.1.0, 2.1.1, and 2.2.0
* Add support for Kafka 2.3.1
* Improved Kafka rolling update
* Improved Kafka Exporter Grafana dashboard
* Add sizeLimit option to ephemeral storage
* Add `schedulerName` to `podTemplate`
* Allow overriding the auto-detected Kubernetes version
* Garbage Collection (GC) logging disabled by default
* Providing PKCS12 truststore and password in the cluster and clients CA certificates Secrets
* Providing PKCS12 keystore and password in the TLS based KafkaUser related Secret

For more details and the upgrade procedure, go to: https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.15.0-rc1

The main changes in the Bridge include:
* Added support for Jaeger tracing
* Various bug fixes.

Jakub


[ANNOUNCE] Strimzi Kafka OAuth library 0.2.0 release

Jakub Scholz
 

Hi,

The 0.2.0 version of the Strimzi Kafka OAuth library for using OAuth authentication in Kafka clients and brokers has been released and should be available in the Maven repositories. The main improvement is improved support for other OAuth servers than just Keycloak. For more information, see the release page on GitHub: https://github.com/strimzi/strimzi-kafka-oauth/releases/tag/0.2.0

Thanks to everyone who contributed to this release.

Thanks & Regards
Jakub

81 - 100 of 106