For monitoring kafka, one can use logs, the Kafka APIs to get metadata like topics/partitions/offsets and JMX. The kafka module in metricbeat uses the Kafka API, but the metrics listed here are exported via JMX. Metricbeat has generic collectors via JMX (via Jolokia), HTTP and Prometheus endpoints. By running kafka with the Jolokia or Prometheus agent, you can configure metricbeat to also collect these stats.