# Kafka + Metricbeat error:Failed to fetch offset for (topic, partition)

**URL:** <https://discuss.elastic.co/t/kafka-metricbeat-error-failed-to-fetch-offset-for-topic-partition/305777>\
**Category:** Beats\
**Tags:** docker, metricbeat\
**Created:** [May 27, 2022, 8:15am UTC](https://discuss.elastic.co/t/kafka-metricbeat-error-failed-to-fetch-offset-for-topic-partition/305777 "2022-05-27T08:15:08Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![ekeras](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ekeras/32/101676_2.png) [@ekeras](https://discuss.elastic.co/u/ekeras)\
**Post date:** [May 27, 2022, 8:15am UTC](https://discuss.elastic.co/t/kafka-metricbeat-error-failed-to-fetch-offset-for-topic-partition/305777/1 "2022-05-27T08:15:08Z")

</div>

I am running elastic stack by [this repository](https://github.com/deviantony/docker-elk). I just changed the network elastic stack is being hosted on, so it would be on the same network as Kafka is.

To set up monitoring kafka metrics with Elastic, I have followed [this tutorial](https://www.elastic.co/blog/how-to-monitor-containerized-kafka-with-elastic-observability).

I have my kafka set up on cloud machine.

Kafka zookeeper and broker:

```auto
  zookeeper:
    image: confluentinc/cp-zookeeper:7.0.0
    hostname: zookeeper
    container_name: zookeeper
    ports:
      - "2181:2181"
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000
      KAFKA_OPTS: "-Dzookeeper.4lw.commands.whitelist=srvr,mntr,ruok,conf,cons"
    labels:
      - co.elastic.metrics/module=zookeeper
      - co.elastic.metrics/hosts=zookeeper:2181
      - co.elastic.metrics/metricsets=mntr,server

  broker:
    image: confluentinc/cp-server:7.0.0
    hostname: broker
    container_name: broker
    volumes:
      - ./jolokia-jvm-1.6.2-agent.jar:/home/appuser/jolokia.jar
    depends_on:
      - zookeeper
    ports:
      - "9092:9092"
      - "9101:9101"
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://broker:29092,PLAINTEXT_HOST://localhost:9092
      KAFKA_METRIC_REPORTERS: io.confluent.metrics.reporter.ConfluentMetricsReporter
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
      KAFKA_CONFLUENT_LICENSE_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_CONFLUENT_BALANCER_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
      KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
      KAFKA_CONFLUENT_SCHEMA_REGISTRY_URL: http://schema-registry:8081
      CONFLUENT_METRICS_REPORTER_BOOTSTRAP_SERVERS: broker:29092
      CONFLUENT_METRICS_REPORTER_TOPIC_REPLICAS: 1
      CONFLUENT_METRICS_ENABLE: 'true'
      CONFLUENT_SUPPORT_CUSTOMER_ID: 'anonymous'
      KAFKA_JMX_OPTS: '-javaagent:/home/appuser/jolokia.jar=port=9101,host=broker -Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false'
    labels:
      - co.elastic.logs/module=kafka
      - co.elastic.metrics/module=kafka
      - co.elastic.metrics/metricsets=partition,consumergroup
      - co.elastic.metrics/hosts='$${data.container.name}:9092'
      - co.elastic.metrics/1.module=kafka
      - co.elastic.metrics/1.metricsets=broker
      - co.elastic.metrics/1.hosts='$${data.container.name}:9101'

```

Error I am getting from metricbeat:

```auto
metricbeat_1 | {"log.level":"error","@timestamp":"2022-05-27T08:13:26.038Z","log.origin":{"file.name":"consumergroup/query.go","file.line":119},"message":"failed to fetch offset for (topic, partition): ('_confluent-metrics', 10)","service.name":"metricbeat","ecs.version":"1.6.0"}
metricbeat_1 | {"log.level":"error","@timestamp":"2022-05-27T08:13:26.040Z","log.origin":{"file.name":"consumergroup/query.go","file.line":119},"message":"failed to fetch offset for (topic, partition): ('_confluent-metrics', 8)","service.name":"metricbeat","ecs.version":"1.6.0"}
metricbeat_1 | {"log.level":"error","@timestamp":"2022-05-27T08:13:26.041Z","log.origin":{"file.name":"consumergroup/query.go","file.line":119},"message":"failed to fetch offset for (topic, partition): ('_confluent-metrics', 7)","service.name":"metricbeat","ecs.version":"1.6.0"}
metricbeat_1 | {"log.level":"error","@timestamp":"2022-05-27T08:13:26.043Z","log.origin":{"file.name":"consumergroup/query.go","file.line":119},"message":"failed to fetch offset for (topic, partition): ('_confluent-metrics', 2)","service.name":"metricbeat","ecs.version":"1.6.0"}

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 24, 2022, 10:16am UTC](https://discuss.elastic.co/t/kafka-metricbeat-error-failed-to-fetch-offset-for-topic-partition/305777/2 "2022-06-24T10:16:07Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.