|
[ANNOUNCE] [RELEASE] Strimzi Kafka Bridge 0.21.5
Jakub Scholz
New version 0.21.5 of Strimzi Kafka Bridge has been released. The main changes in this release are: * Support for ppc64le platform * Documentation improvements * Dependency updates For more details, go to https://github.com/strimzi/strimzi-kafka-bridge/releases/tag/0.21.5 Thanks to everyone who contributed to to this releases! Regards Strimzi team |
||
|
|
||
|
RC1 of Strimzi Kafka Bridge 0.21.5
Jakub Scholz
Release Candidate 1 of Strimzi Kafka Bridge the 0.21.5 is now available for testing. The main changes since 0.21.4 are: * Support for ppc64le platform * Documentation improvements * Dependency updates More details and a full list of changes can be found on the GitHub release page: https://github.com/strimzi/strimzi-kafka-bridge/releases/tag/0.21.5-rc1 Any feedback can be provided on the mailing list, on Slack or as a GitHub issue. Thanks & Regards Strimzi team |
||
|
|
||
|
Draft Strimzi proposal to CNCF "incubation" stage
Paolo Patierno
We would like to propose Strimzi to the "incubation" stage in CNCF. Here there is a PR with a draft proposal on my own GitHub profile to allow everyone to take a look at it and engage. We would really love to have feedback and comments from the community before
submitting the official PR to the CNCF TOC. It will be really appreciated!
https://github.com/ppatierno/toc/pull/1
The Strimzi maintainers
Paolo Patierno
Principal Software Engineer @ Red Hat
Microsoft MVP on Azure |
||
|
|
||
|
[ANNOUNCE] [RELEASE] Strimzi Test Container Operators 0.101.0 is out
Maros Orsak
Strimzi Test Container 0.101.0 has been released. Main changes since the 0.100.0 release include: - adding the ability to configure ZooKeeper container via zookeeper.properties - adding the ability to connect to internal ZooKeeper inside StrimziKafkaContainer using the getInternalZooKeeperConnect() method - constructor for an only specified number of brokers - better logs description - adds support for ppc64le architecture **Github links** - https://github.com/strimzi/test-container/releases/tag/0.101.0 - https://github.com/strimzi/test-container-images/releases/tag/0.101.0 |
||
|
|
||
|
[ANNOUCE] [Release Candidate 2] Strimzi test containers 0.101.0
Maros Orsak
Release candidate 2 of Strimzi test containers 0.101.0 is now available for testing. The main changes from version 0.101.0-rc1 in this release include :
Images used by Strimzi test container (no change in images) To test the Maven artefacts which are part of this release, use the staging repository by including the following in your pom.xml: Best regards, |
||
|
|
||
|
[ANNOUCE] [Release Candidate 1] Strimzi test containers 0.101.0
Maros Orsak
Release candidate 1 of Strimzi test containers 0.101.0 is now available for testing. The main changes from version 0.100.0 in this release include : - adding the ability to configure ZooKeeper container via `zookeeper.properties` - adding the ability to connect to internal ZooKeeper inside `StrimziKafkaContainer` using `getInternalZooKeeperConnect()` method - constructor for an only specified number of brokers - better logs description - adds support for `ppc64le` architecture Images used by Strimzi test container To test the Maven artefacts which are part of this release, use the staging repository by including the following in your pom.xml: Best regards, |
||
|
|
||
|
Re: Need help setting UP Strimzi Kafka Cluster with Cruise Control
Jakub Scholz
You have to do it manually. Kafka is not something that is normally scaled up or down very often. It is a stateful application with partitions and their replicas having a fixed assignment to a specific broker. Scaling the cluster up and down means reassigning the partitions between the nodes and since the partitions often contain very large amounts of data, these data need to be transferred. And that has of course its cost (it often takes a long time, generates financial costs etc.). So usually, you would scale the cluster up for example as you onboard new applications or as your applications grow. But you would for example not scale it up for a few minutes because there seems to be some traffic peak. So normally it is more the result of some long term planning rather than some quick decision of the HPA. Thanks & Regards Jakub On Wed, Feb 23, 2022 at 9:48 PM Ranjeet Ranjan <ranjeet@...> wrote:
|
||
|
|
||
|
Re: Need help setting UP Strimzi Kafka Cluster with Cruise Control
Ranjeet Ranjan
Hi Jakub, I have set up Strimzi Kafka Cluster and Operator. I have still one doubt that I did not answer, do I need to scale the Kafka cluster and data balance up/down manually or will it scale up/down based on the Kubernetes HPA? Your help will be much appreciated. On Tue, Feb 22, 2022 at 11:47 PM Jakub Scholz <jakub@...> wrote:
|
||
|
|
||
|
Re: Need help setting UP Strimzi Kafka Cluster with Cruise Control
Jakub Scholz
No, we do not have a Helm Chart for creating the Kafka custom resource. Thanks & Regards Jakub On Tue, Feb 22, 2022 at 6:40 PM Ranjeet Ranjan <ranjeet@...> wrote:
|
||
|
|
||
|
Re: Need help setting UP Strimzi Kafka Cluster with Cruise Control
Ranjeet Ranjan
Hi Jakub, Thank you for your response. DO we have a helm as well for creating Kafka Cluster? On Tue, Feb 22, 2022 at 11:08 PM Jakub Scholz <jakub@...> wrote:
|
||
|
|
||
|
Re: Need help setting UP Strimzi Kafka Cluster with Cruise Control
Jakub Scholz
HI Ranjeet, The Helm Chart installs only the operator for managing the Kafka clusters. Next, you need to create the Kafka custom resource describing the Kafka cluster you want to have, its configuration etc. The docs (https://strimzi.io/documentation/) cover all the different aspects which you can configure. We have also all kind of examples on GitHub: https://github.com/strimzi/strimzi-kafka-operator/tree/main/examples Thanks & Regards Jakub On Tue, Feb 22, 2022 at 6:33 PM Ranjeet Ranjan <ranjeet@...> wrote:
|
||
|
|
||
|
Need help setting UP Strimzi Kafka Cluster with Cruise Control
Ranjeet Ranjan
Hi Team,
Let me introduce myself Ranjeet Ranjan, CEO & Founder Adsizzler Media. We have recently launched one new Ad Tech Startup Named http://adosiz.com/ We wanted to install but need help setting UP Strimzi Kafka Cluster with Cruise Control for Auto Scale. I have downloaded helm from https://strimzi.io/downloads/. I followed the below steps to install. Step -1: helm install strimzi-kafka-operator strimzi-kafka-operator -n test But it only installed strimzi-cluster-operator-5f56c65c5d-4pphv Now please let know how Kafka Cluster pod will be created. Thanks Ranjeet Ranjan
|
||
|
|
||
|
Re: adding annotation to service account
Jakub Scholz
Hi Amit, You can use the `template` section to declaratively customize annotations: https://strimzi.io/docs/operators/latest/full/configuring.html#assembly-customizing-kubernetes-resources-str ... for Service Accounts, from 0.27.0 with the ServiceAccountPatching feature gate enabled, it should be possible to use that even for existing clusters / service accounts. In earlier versions (or with the ServiceAccountPatching feature gate disables) it will be taken into account only when a new Service account is created. In that case, you can also annotate it simply with `kubectl annotate ...`. Thanks & Regards Jakub On Mon, Feb 21, 2022 at 10:54 AM <amit.cahanovich@...> wrote: Hi, |
||
|
|
||
|
adding annotation to service account
amit.cahanovich@...
Hi,
I work with strimzi (as kafka connect) on eks. I would like to add to the service account aws role annotation to s3 (something like: eks.amazonaws.com/role-arn: arn:aws:iam::xxxxxxx:role/s3-read-role). Is there a trivial way to do it? Thanks, Amit |
||
|
|
||
|
[ANNOUNCE] [RELEASE] Strimzi Kafka Operators 0.28.0
Jakub Scholz
Strimzi Kafka Operators 0.28.0 has been released. The main changes in this release include: * Add support for Kafka 3.1.0; remove Kafka 2.8.0 and 2.8.1 * Add support for `StrimziPodSet` resources (disabled by default through the `UseStrimziPodSets` feature gate) * Support custom authentication mechanisms in Kafka listeners * Intra-broker disk balancing using Cruise Control * Added the option `createBootstrapService` in the Kafka Spec to disable the creation of the bootstrap service for the Load Balancer Type Listener. It will save the cost of one load balancer resource, specially in the public cloud. * Add support for disabling the FIPS mode in OpenJDK * Fix renewing your own CA certificates For more details and installation files, go to https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.28.0 We have also created a video about the changes in this release which you can watch on our YouTube channel: https://youtu.be/PZKbrDUU1zo Important: This release supports only the API version v1beta2 and CRD version apiextensions.k8s.io/v1. If upgrading from Strimzi 0.22, migration to v1beta2 needs to be completed for all Strimzi CRDs and CRs before the upgrade to 0.28 is done! If upgrading from Strimzi version earlier than 0.22, you need to first install the CRDs from Strimzi 0.22 and complete the migration to v1beta2 for all Strimzi CRDs and CRs before the upgrade to 0.28 is done! For more details about the CRD upgrades, see the documentation. Thanks to everyone who contributed to these releases! Thanks & Regards Strimzi team |
||
|
|
||
|
RC1 of Strimzi Kafka Operators 0.28.0 is available for testing
Jakub Scholz
Release candidate 1 of Strimzi Kafka Operators 0.28.0 is now available for testing. The main changes in this release include: * Add support for Kafka 3.1.0; remove Kafka 2.8.0 and 2.8.1 * Add support for `StrimziPodSet` resources (disabled by default through the `UseStrimziPodSets` feature gate) * Support custom authentication mechanisms in Kafka listeners * Intra-broker disk balancing using Cruise Control * Added the option `createBootstrapService` in the Kafka Spec to disable the creation of the bootstrap service for the Load Balancer Type Listener. It will save the cost of one load balancer resource, specially in the public cloud. * Add support for disabling the FIPS mode in OpenJDK * Fix renewing your own CA certificates For more details and installation files, go to https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.28.0-rc1 Any feedback can be provided on the Strimzi mailing list, on the #strimzi Slack channel on CNCF Slack or as a GitHub issue. Thanks & Regards Strimzi team |
||
|
|
||
|
[ANNOUNCE] [RELEASE] Strimzi Kafka Bridge 0.21.4
Jakub Scholz
New version 0.21.4 of Strimzi Kafka Bridge has been released. The main changes in this release are: * Dependency updates (Configuration providers, Vert.x, Netty, Oauth client and more) * Add support for disabling the FIPS mode in OpenJDK * Add transactions `isolation.level` configuration parameter on consumer creation * Support for s390x platform For more details, go to https://github.com/strimzi/strimzi-kafka-bridge/releases/tag/0.21.4 Thanks to everyone who contributed to any of these releases! Regards Strimzi team |
||
|
|
||
|
Re: Broker Hostname mismatch error seen with aiokafka APIs
Jakub Scholz
You are using the external loadbalancer listener with internal service name. That is causing the problem. You should either switch to one of the internal interfaces - e.g. the on on port 9093. Or you should use the proper loadbalancer bootstrap address which you can find in the status section of the Kafka custom resource (`kubectl get kafka -o yaml` should show it for you). In general, if your app runs inside the same Kubernetes, using the internal listener on port 9093 would be the right way to go => it should be cheaper and more performant than going through the loadbalancer. That should be used by apps outside your Kube cluster. Jakub On Fri, Feb 4, 2022 at 10:49 AM <udaykumartj@...> wrote:
|
||
|
|
||
|
Broker Hostname mismatch error seen with aiokafka APIs
udaykumartj@...
Hi all,
I am getting the below error for the code (which uses aiokafka APIs) that i have pasted below (also provided kafka resource yaml snippet). Instead of kafka broker hostname, if i give IP address, it works. Not sure why kafka broker (dns) hostname is not working. Please help. Unable connect to "strimzi-kafka-kafka-external-bootstrap.kafka.svc.cluster.local:9094": [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: Hostname mismatch, certificate is not valid for 'strimzi-kafka-kafka-external-bootstrap.kafka.svc.cluster.local'. (_ssl.c:1129) Note: If i use confluent_kafka APIs (Consumer, Producer), i dont see this issue. Code snippet: context = create_ssl_context( cafile='/etc/vcerts/cluster/ca.crt',
certfile='/etc/vcerts/client/user.crt',
keyfile='/etc/vcerts/client/user.key',
)
consumer = AIOKafkaConsumer(
my_topic, bootstrap_servers='strimzi-kafka-kafka-external-bootstrap.kafka.svc.cluster.local:9094',
auto_offset_reset='latest',
group_id=group_id,
security_protocol="SSL",
ssl_context=context)
await consumer.start() <=== This line throws the above error.
Kafka spec: apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
name: strimzi-kafka
spec:
kafka:
version: 3.0.0
replicas: 3
listeners:
- name: plain
port: 9092
type: internal
tls: false
- name: tls
port: 9093
type: internal
tls: true
authentication:
type: tls
- name: external
port: 9094
type: loadbalancer
tls: true
authentication:
type: tls
template:
clusterCaCert:
metadata:
....
|
||
|
|
||
|
Broker Hostname mismatch error seen with aiokafka APIs
udaykumartj@...
Hi all,
I am getting the below error for the code (which uses aiokafka APIs) that i have pasted below (also provided kafka resource yaml snippet). Instead of kafka broker hostname, if i give IP address, it works. Not sure why kafka broker (dns) hostname is not working. Please help. Unable connect to "strimzi-kafka-kafka-external-bootstrap.kafka.svc.cluster.local:9094": [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: Hostname mismatch, certificate is not valid for 'strimzi-kafka-kafka-external-bootstrap.kafka.svc.cluster.local'. (_ssl.c:1129) Note: If i use confluent_kafka APIs (Consumer, Producer), i dont see this issue. Code snippet: context = create_ssl_context( cafile='/etc/vcerts/cluster/ca.crt',
certfile='/etc/vcerts/client/user.crt',
keyfile='/etc/vcerts/client/user.key',
)
consumer = AIOKafkaConsumer(
my_topic, bootstrap_servers='strimzi-kafka-kafka-external-bootstrap.kafka.svc.cluster.local:9094',
auto_offset_reset='latest',
group_id=group_id,
security_protocol="SSL",
ssl_context=context)
await consumer.start() <=== This line throws the above error.
Kafka spec: apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
name: strimzi-kafka
spec:
kafka:
version: 3.0.0
replicas: 3
listeners:
- name: plain
port: 9092
type: internal
tls: false
- name: tls
port: 9093
type: internal
tls: true
authentication:
type: tls
- name: external
port: 9094
type: loadbalancer
tls: true
authentication:
type: tls
template:
clusterCaCert:
metadata:
....
|
||
|
|
