APM server exits if Kafka output is higher than version 2.0

Elastic Observability APM
,
1

APM Server version:
Tested 7.13 and 7.15, fails on both

Original install method (e.g. download page, yum, deb, from source, etc.) and version:
Docker container with the below dockerfile

FROM docker.elastic.co/apm/apm-server:latest
COPY apm-kafka.yml /usr/share/apm-server/apm-server.yml
USER root
RUN chown root:apm-server /usr/share/apm-server/apm-server.yml
USER apm-server

Fresh install or upgraded from other version?
Fresh install for a proof of concept

Is there anything special in your setup? For example, are you using the Logstash or Kafka outputs? Are you using a load balancer in front of the APM Servers? Have you changed index pattern, generated custom templates, changed agent configuration etc.

I am using Kafka output with the following configs in apm-server.yml

output.kafka:
  enabled: true
  hosts: ["kafka:9092"]
  topic: apm
  partition.round_robin:
    reachable_only: false
  version: '2.1.0'

Description of the problem including expected versus actual behavior. Please include screenshots (if relevant):

Reading through the documentation under compatibility I find the following text "This output works with all Kafka versions in between 0.11 and 2.2.2" although under the Version section there is a mismatch with the version numbers above " Valid values are all kafka releases in between 0.8.2.0 and 2.0.0 ."
Clearly either the compatibility section has incorrect versions numbers or the code was not shipped with support for versions higher than 2.0.

Also I want to use a higher version than what is currently supported. Currently it would be 2.4.0. Do you guys foresee any potential issues here?
I can get messages in my topic with my version (2.4.0) just cannot configure the yml to match it.

Steps to reproduce:
1.Download APM image and build with above apm-server.yml
2. Run a container and wait for it to fail with version error

Provide logs and/or server output (if relevant):
Exiting: error initializing publisher: unknown/unsupported kafka vesion '2.1.0' accessing 'output.kafka.version' (source:'apm-server.yml')

2

@joao-subtil welcome to the forum! Sorry about the delay in responding to you.

Also I want to use a higher version than what is currently supported. Currently it would be 2.4.0. Do you guys foresee any potential issues here?

I'd suggest not specifying output.kafka.version at all -- the default (1.0.0) will work for newer brokers too. Increasing the version permits additional Kafka features, but we don't use them.

If you don't specify the version, then the server will work with recent versions of Kafka. I just tested APM Server v7.15.2 with Confluent Platform 6.2.0 (Kafka 2.8.x), and it works well.

I'll see what we can do about clarifying the docs.

3

Thank you for the reply. I was a bit confused and thought the kafka versioning had to match due to something under the hood.

You can close this thread if you want.
thanks!

1 Like
4

I was a bit confused and thought the kafka versioning had to match due to something under the hood.

Totally understandable. I opened docs: clarify kafka output config and version compatibility · Issue #29310 · elastic/beats · GitHub in response to this thread.

5

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.