Can't monitor kafka module metrics like producer, consumer, broker in 6.8.1 and later?

[17earlgrey]

• https://www.elastic.co/guide/en/beats/metricbeat/master/metricbeat-module-kafka.html
  If you look at the link here, it seems that earlier versions provided the functionality (consumer, producer, broker) in beta.

• https://www.elastic.co/guide/en/beats/metricbeat/6.8/metricbeat-module-kafka.html
  However, if you look at version 6.8 and above, the setting is gone as above. (Only partitions and consumer groups available)

When I put it in the configuration I get this error “metricset 'kafka / producer' is not registered, metricset not found”.

Now I wonder if it doesn't provide consumer, producer, or broker metrics.

Version

• Kafka : 5.3.1-ccs (Docker Image : confluentinc/cp-kafka:5.3.1)
• Metricbeat : 6.8.1

[Kaiyan_Sheng]

Hello @17earlgrey, with the latest Metricbeat kafka module, we have broker, consumer, consumergroup, partition and producer metricsets. Please see https://www.elastic.co/guide/en/beats/metricbeat/master/metricbeat-module-kafka.html#_metricsets_27 for more details.