# Can we get network metrics of kafka through packetbeat? **URL:** \ **Category:** Beats\ **Tags:** packetbeat\ **Created:** [May 28, 2016, 8:32am UTC](https://discuss.elastic.co/t/can-we-get-network-metrics-of-kafka-through-packetbeat/51214 "2016-05-28T08:32:42Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![natarajan\_k](https://avatars.discourse-cdn.com/v4/letter/n/f14d63/32.png) [@natarajan\_k](https://discuss.elastic.co/u/natarajan_k)\ **Post date:** [May 28, 2016, 8:32am UTC](https://discuss.elastic.co/t/can-we-get-network-metrics-of-kafka-through-packetbeat/51214/1 "2016-05-28T08:32:43Z")
i have kafka and packetbeat in same machine of ubuntu distribution. i can get the network statistics of 80 and 53 port from packetbeat but i would like to know is there any way we can get the kafka traffic. i have seen the option of the process in packetbeat.yml which makes me to try with kafka. here is my configuration: packetbeat.yml: > > **interfaces** : > > device: any > > **protocols** : > > dns: > > ports: [53] > > include\_authorities: true > > include\_additionals: true http: > > ``` > > ports: [80, 8080, 8000, 5000, 8002] > > > > ``` procs: > > ``` > > enabled: true > > - process: kafk-1 > > cmdline_grep: QuorumPeerMain > > > > ``` output: > > ``` > > logstash: > > hosts: ["fqdn:5048"] > > > > ``` shipper: > > ``` > > tags: ["kafka-01"] > > ignore_outgoing: false > > > > ``` logging: > > ``` > > to_files: true > > files: > > > > ``` > > > > path: /var/log/packbeat > > name: packbeat.log > > rotateeverybytes: 10485760 # = 10MB i have started the zookeeper service from the port 2181 and kafka server with 9093. process name i referred from the path /proc/pid/cmdline. zookeeper has cmdline: /usr/bin/java-cp/etc/zookeeper/conf:/usr/share/java/jline.jar:/usr/share/java/log4j-1.2.jar:/usr/share/java/xercesImpl.jar:/usr/share/java/xmlParserAPIs.jar:/usr/share/java/netty.jar:/usr/share/java/slf4j-api.jar:/usr/share/java/slf4j-log4j12.jar:/usr/share/java/zookeeper.jar-Dcom.sun.management.jmxremote-Dcom.sun.management.jmxremote.local.only=false-Dzookeeper.log.dir=/var/log/zookeeper-Dzookeeper.root.logger=INFO,ROLLINGFILEorg.apache.zookeeper.server.quorum.QuorumPeerMain/etc/zookeeper/conf/zoo.cfg kafka server has cmdline: java-Xmx1G-Xms1G-server-XX:+UseG1GC-XX:MaxGCPauseMillis=20-XX:InitiatingHeapOccupancyPercent=35-XX:+DisableExplicitGC-Djava.awt.headless=true-Xloggc:/home/kafka/kafka/bin/../logs/kafkaServer-gc.log-verbose:gc-XX:+PrintGCDetails-XX:+PrintGCDateStamps-XX:+PrintGCTimeStamps-Dcom.sun.management.jmxremote-Djava.rmi.server.hostname=(publicaddr)-Dcom.sun.management.jmxremote.authenticate=false-Dcom.sun.management.jmxremote.ssl=false-Dkafka.logs.dir=/home/kafka/kafka/bin/../logs-Dlog4j.configuration=file:bin/../config/log4j.properties-cp:/home/kafka/kafka/bin/../libs/\*kafka.Kafkaconfig/server-1.properties i have mentioned the below for the zookeeper in packetbeat.yml: cmdline\_grep: QuorumPeerMain question: 1. after the above packetbeat configuration, i have started the packetbeat service in debug mode, i dont see network socket from the process name into elasticsearch nor i am receiving in logstash. 2.i don't have idea how to mention cmdline\_grep process name for kafka server from the /proc/pid/cmdline. i also tried mentioning the process name from /proc/pid/comm but it was given "java". can we mention java as process name in packetbeat.yml? your help would be much appreciated. thanks in advance. ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [May 30, 2016, 1:25pm UTC](https://discuss.elastic.co/t/can-we-get-network-metrics-of-kafka-through-packetbeat/51214/2 "2016-05-30T13:25:18Z")
packetbeat is a passive protocol analyzer, reassembling tcp streams from network packets. No kafka protocol analyzer is implemented yet (but shouldn't be too hard). Alternatively one can use metricbeat kafka and zookeeper modules to pull some stats right from kafka. > 1. after the above packetbeat configuration, i have started the packetbeat service in debug mode, i dont see network socket from the process name into elasticsearch nor i am receiving in logstash. Anything in logs? Logstash reachable? Please format logs, files and command lines using 3 backticks '`' to make them readable and keep format. I can not tell if your config file is correct or not without formatting. > 2.i don't have idea how to mention cmdline\_grep process name for kafka server from the /proc/pid/cmdline. i also tried mentioning the process name from /proc/pid/comm but it was given "java". can we mention java as process name in packetbeat.yml? I don't quite understand what kind of problem you try to solve here. ---
**Author:** ![allenmchan](https://avatars.discourse-cdn.com/v4/letter/a/ec9cab/32.png) [@allenmchan](https://discuss.elastic.co/u/allenmchan)\ **Post date:** [June 15, 2016, 5:21am UTC](https://discuss.elastic.co/t/can-we-get-network-metrics-of-kafka-through-packetbeat/51214/3 "2016-06-15T05:21:36Z")
Kafka has a lot of JMX metrics that might solve some of your requirements. > **[Apache Kafka](http://kafka.apache.org/documentation.html#monitoring)** > > Apache Kafka: A Distributed Streaming Platform. ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [June 15, 2016, 8:30am UTC](https://discuss.elastic.co/t/can-we-get-network-metrics-of-kafka-through-packetbeat/51214/4 "2016-06-15T08:30:11Z")
There is also a communitybeat collecting kafka jmx stats: [https://github.com/gingerwizard/kafkabeat](https://github.com/gingerwizard/kafkabeat) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 9:51pm UTC](https://discuss.elastic.co/t/can-we-get-network-metrics-of-kafka-through-packetbeat/51214/5 "2017-07-05T21:51:11Z")