Adding annotations and limits of kafka connect created pods
|
alonisser@...
Trying to add pod annotations (for our log connector) and resource request/limits to the KafkaConnect created resources
What should I add in the resource so it would create the pods/deployment with them(I suspect I already sent this message but can't find it now) |
|
|
|
Jakub Scholz
Hi, For specifying the resources, you can do something like this: # ... resources: requests: memory: 1Gi cpu: 500m limits: memory: 2Gi cpu: 1000m # ... For the annotations, it depends what exactly do you want to annotate. But basically you can use this feature: https://strimzi.io/docs/latest/full.html#assembly-customizing-deployments-str Your YAML would look something like this: kind: KafkaConnect metadata: name: my-connect # ... spec: # ... template: deployment: metadata: annotations: myanno: myvalue pod: metadata: annotations: myanno: myvalue # ... Thanks & Regards Jakub On Sun, Feb 23, 2020 at 10:42 AM <alonisser@...> wrote: Trying to add pod annotations (for our log connector) and resource request/limits to the KafkaConnect created resources |
|
|
|
alonisser@...
Thanks I would try that! I was missing the pod/deployment sub level
|
|
|
|
alonisser@...
Thanks, following your advice I've added:
template: And it didn't work and I saw errors in the pods of the operator |
|
|
|
Jakub Scholz
Yeah, the annotations are strings. So you need to wrap it into quotes to make it a string. Without them it will be interpreted as array. On Fri, Feb 28, 2020 at 2:04 PM <alonisser@...> wrote: Thanks, following your advice I've added: |
|
|
|
amit.cahanovich@...
Hi,
i have the following configuration: apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnect
metadata:
name: my-connect-cluster
namespace: kafka
annotations:
strimzi.io/use-connector-resources: "true"
spec:
template:
pod:
metadata:
annotations:
ad.datadoghq.com/kafka-connect-container-name.logs: '[{"type":"file", "source":"java","sourcecategory":"sourcecode", "service":"kafka-connect"}]'
ad.datadoghq.com/container.check_names: '["openmetrics"]'
ad.datadoghq.com/container.init_configs: '[{}]'
ad.datadoghq.com/container.instances: '[{"prometheus_url": "http://%%host%%:9404/metrics","namespace": "kafka","metrics": ["kafka_connect_connector_task_batch_size_avg"],"type_overrides": {"kafka_connect_incoming_byte_total": "gauge"}}]'
image:
i keep on getting: error: error validating "kafka-connect-prometeus.yaml": error validating data: ValidationError(KafkaConnect): unknown field "template" in io.strimzi.kafka.v1beta2.KafkaConnect; if you choose to ignore these errors, turn validation off with --validate=false any advise on it (when removing it. all work smoothly...) Thanks, Amit |
|
|
|
Jakub Scholz
The YAML you shared has no indentation. If that is how it really looks, it is indeed invalid. If the indentation was list during copy pasting, nobody can say what the original indentation was and what would be the error. Thanks & Regards Jakub On Mon, Jan 31, 2022 at 4:08 PM <amit.cahanovich@...> wrote: Hi, |
|
|
|
amit.cahanovich@...
for reference,
The following is kafkaconnect that is integrated with datadog (named my-connect-cluster) apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnect
metadata:
name: my-connect-cluster
namespace: kafka
annotations:
strimzi.io/use-connector-resources: "true"
spec:
image: amitca71/strimzi-connect:0.27.1-kafka-2.8.1
version: 2.8.1
replicas: 3
bootstrapServers: "${bootstrat_servers}"
template:
deployment:
metadata:
annotations:
reloader.stakater.com/auto: "true"
pod:
metadata:
annotations:
ad.datadoghq.com/my-connect-cluster-connect.check_names: '["openmetrics"]'
ad.datadoghq.com/my-connect-cluster-connect.init_configs: '[{}]'
ad.datadoghq.com/my-connect-cluster-connect.instances: '[{"prometheus_url": "http://%%host%%:%%port%%/metrics","namespace": "strimzi_kafka_connect","metrics": ["kafka_connect_connector_task_batch_size_avg", "kafka_admin_client_node_incoming_byte_total", "kafka_connect_worker_connector_paused_task_count","kafka_connect_version_info"]}]'
|
|
|