# Packetbeat data not going from ES to Kibana **URL:** \ **Category:** Beats\ **Tags:** packetbeat\ **Created:** [September 7, 2015, 7:08am UTC](https://discuss.elastic.co/t/packetbeat-data-not-going-from-es-to-kibana/28757 "2015-09-07T07:08:46Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![syn\_](https://avatars.discourse-cdn.com/v4/letter/s/6bbea6/32.png) [@syn\_](https://discuss.elastic.co/u/syn_)\ **Post date:** [September 7, 2015, 7:08am UTC](https://discuss.elastic.co/t/packetbeat-data-not-going-from-es-to-kibana/28757/1 "2015-09-07T07:08:46Z")
Hi guys, Not sure if this is the best category, as it may be either an ElasticSearch or Kibana issue, but since I followed the guide found here: [https://www.elastic.co/guide/en/beats/packetbeat/current/packetbeat-getting-started.html](https://www.elastic.co/guide/en/beats/packetbeat/current/packetbeat-getting-started.html) I think it's fair to create it here. As mentioned, I followed the guide provided by Elastic, skipping the ES and Kibana installs as they were already installed. I tried installing Packetbeat on the ELK server, and also another server hosting mongodb. When I followed the step which asks you to test it via creating a HTTP request ( "curl [http://www.elastic.co/](http://www.elastic.co/) \> /dev/null" ), I did this, and then did the curl ( "curl -XGET '[http://localhost:9200/packetbeat-\*/\_search?pretty](http://localhost:9200/packetbeat-*/_search?pretty)' ") however no HTTP queries were returned by Elastic. I thought it was weird, so I installed it on the server hosting MongoDB, and again didn't see the HTTP query. What I did see however, was data from my MongoDB server - so this indicated it was working to me . Here's an example: ubuntu@proxy01:/var/log/elasticsearch$ curl -XGET '[http://localhost:9200/packetbeat-\*/\_search?pretty](http://localhost:9200/packetbeat-*/_search?pretty)' { "took" : 11, "timed\_out" : false, "\_shards" : { "total" : 20, "successful" : 20, "failed" : 0 }, "hits" : { "total" : 628816, "max\_score" : 1.0, "hits" : [ { "\_index" : "packetbeat-2015.09.04", "\_type" : "mongodb", "\_id" : "AU-WuXSOCLSXQAff\_ybn", "\_score" : 1.0, "\_source":{"bytes\_in":151,"bytes\_out":90,"client\_ip":"10.2.194.215","client\_port":45066,"client\_proc":"","client\_server":"mongo-primary.QA.DEV.LOCAL","count":1,"ip":"10.2.194.216","method":"otherCommand","mongodb":{"cursorId":0,"fullCollectionName":"local.$cmd","numberReturned":1,"numberToReturn":1,"numberToSkip":0,"startingFrom":0},"port":27017,"proc":"","query":"local.$cmd.otherCommand().limit(1)","resource":"local.$cmd","responsetime":0,"server":"","shipper":"mongo-primary.QA.DEV.LOCAL","status":"OK","timestamp":"2015-09-04T04:59:38.896Z","type":"mongodb"} When I continued the guide, following the steps for creating the packetbeat-\* index pattern within Kibana, I couldn't see any data. Even after the weekend. So, following some troubleshooting advice with a nice man named "Warkolm" in IRC, I was able to further verify data in ES ubuntu@proxy01:~$ curl localhost:9200/\_cat/indices yellow open logstash-2015.09.04 5 1 167712 0 81.5mb 81.5mb yellow open packetbeat-2015.09.06 5 1 195938 0 25.4mb 25.4mb yellow open logstash-2015.09.02 5 1 243716 0 47mb 47mb yellow open logstash-2015.09.01 5 1 240887 0 45.2mb 45.2mb yellow open packetbeat-2015.09.05 5 1 195951 0 25.5mb 25.5mb yellow open packetbeat-2015.09.04 5 1 143022 0 65.3mb 65.3mb yellow open logstash-2015.08.31 5 1 307528 0 86.8mb 86.8mb yellow open .packetbeat-topology 5 1 2 0 12.8kb 12.8kb yellow open packetbeat-2015.09.07 5 1 83439 0 22mb 22mb yellow open logstash-2015.09.03 5 1 144903 0 64.4mb 64.4mb yellow open logstash-2015.08.12 5 1 18 0 154kb 154kb yellow open logstash-2015.08.28 5 1 182024 0 39.5mb 39.5mb yellow open .kibana 1 1 3 1 14.1kb 14.1kb Output of ES logfile: ubuntu@proxy01:/var/log/elasticsearch$ cat logstash.log [2015-09-07 00:00:02,988][INFO][cluster.metadata] [smoker] [packetbeat-2015.09.07] creating index, cause [auto(bulk api)], shards [5]/[1], mappings [_default_] [2015-09-07 00:00:03,843][INFO][cluster.metadata] [smoker] [packetbeat-2015.09.07] update\_mapping [mongodb] (dynamic) [2015-09-07 06:33:49,082][INFO][cluster.metadata] [smoker] [packetbeat-2015.09.07] update\_mapping [http] (dynamic) And here is a screenshot from Kibana: [http://puu.sh/k2uwU/9319f1c8e8.png](http://puu.sh/k2uwU/9319f1c8e8.png) ![](https://us1.discourse-cdn.com/elastic/original/2X/c/c6b60288cdfcd1aae863fd050aebc273b12735a6.png) At this point i'm quite stumped, so any further advice would be great ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [September 7, 2015, 7:26am UTC](https://discuss.elastic.co/t/packetbeat-data-not-going-from-es-to-kibana/28757/2 "2015-09-07T07:26:16Z")
Just to add, after adding the index filter into KB ad then heading to discover, it just sits there. I note that your hostname is `proxy01`, does that imply there is a proxy between KB and ES? > [@syn\_](#): > > So, following some troubleshooting advice with a nice lady named "Warkolm" in IRC Happy to help 😛 ---
**Author:** ![tudor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tudor/32/3753_2.png) [@tudor](https://discuss.elastic.co/u/tudor)\ **Post date:** [September 7, 2015, 3:26pm UTC](https://discuss.elastic.co/t/packetbeat-data-not-going-from-es-to-kibana/28757/3 "2015-09-07T15:26:54Z")
So data seems to be in Elasticsearch, but it's not shown by Kibana, right? Maybe try opening the developer console in your browser and see if it reports any Javascript errors. ---
**Author:** ![syn\_](https://avatars.discourse-cdn.com/v4/letter/s/6bbea6/32.png) [@syn\_](https://discuss.elastic.co/u/syn_)\ **Post date:** [September 7, 2015, 11:09pm UTC](https://discuss.elastic.co/t/packetbeat-data-not-going-from-es-to-kibana/28757/4 "2015-09-07T23:09:35Z")
I tried deleting and re-creating the index pattern packetbeat-\* with timestamp, no dice though.. @warkolm - no, just the name of the host. And I am so sorry for calling you a lady!!! It was a long day @tudor - there are some js errors relating to "timestamp" being a missing field. How can @timestamp be missing? Isnt it provided by ES? The error occurs each time I click "discover" ![](https://us1.discourse-cdn.com/elastic/original/2X/b/b9ade770434948d6f559853a6fd4137ba72a6148.png) ---
**Author:** ![tudor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tudor/32/3753_2.png) [@tudor](https://discuss.elastic.co/u/tudor)\ **Post date:** [September 8, 2015, 7:22am UTC](https://discuss.elastic.co/t/packetbeat-data-not-going-from-es-to-kibana/28757/5 "2015-09-08T07:22:38Z")
`@timestamp` is not provided by ES, it's just added by Logstash. Packetbeat uses `timestamp`, without the `@`. You should try deleting the index pattern in Kibana and add it again using `timestamp` for the timestamp field. If `timestamp` without the `@` is not offered, I recommend going through: stop packetbeat, delete all packetbeat-\* indexes, make sure the packetbeat template is loaded, start packetbeat. ---
**Author:** ![syn\_](https://avatars.discourse-cdn.com/v4/letter/s/6bbea6/32.png) [@syn\_](https://discuss.elastic.co/u/syn_)\ **Post date:** [September 10, 2015, 1:50am UTC](https://discuss.elastic.co/t/packetbeat-data-not-going-from-es-to-kibana/28757/6 "2015-09-10T01:50:03Z")
@tudor that's exactly what I had done already - I did it again just for good measure and it did offer "timestamp" and not "@timestamp". It's still not showing up in Discover though.. It's kind of weird because the indices do match the packetbeat-\* pattern thats created .. ---
**Author:** ![tudor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tudor/32/3753_2.png) [@tudor](https://discuss.elastic.co/u/tudor)\ **Post date:** [September 10, 2015, 7:12am UTC](https://discuss.elastic.co/t/packetbeat-data-not-going-from-es-to-kibana/28757/7 "2015-09-10T07:12:26Z")
Do you still see the JS exceptions talking about missing `@timestamp`? Maybe we've moved to another error now. ---
**Author:** ![Augustin\_Chen](https://avatars.discourse-cdn.com/v4/letter/a/b5ac83/32.png) [@Augustin\_Chen](https://discuss.elastic.co/u/Augustin_Chen)\ **Post date:** [December 24, 2015, 8:52am UTC](https://discuss.elastic.co/t/packetbeat-data-not-going-from-es-to-kibana/28757/8 "2015-12-24T08:52:52Z")
I have the similar issue, I can see the packetbeat-\* indexes in ES, but the Kibana is now showing any packetbeat events. I use the following stacks: packetbeat: 1.0.0 ES: 2.1.0 Kibana: 4.3.0 This is what I get from the ES. [root@i-6d31fcdc ~]# curl -XGET '[http://10.10.0.65:9200/packetbeat-](http://10.10.0.65:9200/packetbeat-)_/\_search?pretty' { "took" : 20, "timed\_out" : false, "\_shards" : { "total" : 35, "successful" : 35, "failed" : 0 }, "hits" : { "total" : 651145, "max\_score" : 1.0, "hits" : [ { "\_index" : "packetbeat-2015.12.18", "\_type" : "http", "\_id" : "AVG0YK-uqvqHQIKqybYs", "\_score" : 1.0, "\_source":{"@timestamp":"2015-12-18T09:16:53.097Z","beat":{"hostname":"i-1e3ffbae","name":"i-1e3ffbae"},"bytes\_in":455,"bytes\_out":725,"client\_ip":"10.10.1.76","client\_port":11430,"client\_proc":"","client\_server":"","count":1,"direction":"in","http":{"code":200,"content\_length":534,"phrase":"OK"},"ip":"10.10.2.83","method":"GET","params":"","path":"/auth-services/oauth/validatetoken","port":80,"proc":"nginx","query":"GET /auth-services/oauth/validatetoken","real\_ip":"10.10.0.152","request":"GET /auth-services/oauth/validatetoken HTTP/1.1\r\nhost: [auth-services-pw-dev-internal.mse-esp.com](http://auth-services-pw-dev-internal.mse-esp.com)\r\nAccept: text/plain, application/json, application/_+json, _/_\r\nAuthorization: Bearer160e0d7e-611a-443f-97cc-f924af6c5bf0\r\nCache-Control: no-cache\r\nnonce: 963e0180af86a91998cdb5d889ce447e\r\nPragma: no-cache\r\nts: 1450430212862\r\nUser-Agent: Java/1.7.0\_91\r\nX-Forwarded-For: 10.10.0.152\r\nX-Forwarded-Port: 443\r\nX-Forwarded-Proto: https\r\nConnection: keep-alive\r\n\r\n","response":"HTTP/1.1 200 OK\r\nServer: nginx\r\nDate: Fri, 18 Dec 2015 09:16:53 GMT\r\nContent-Type: application/json\r\nContent-Length: 534\r\nConnection: keep-alive\r\nCache-Control: no-store\r\nPragma: no-cache\r\n\r\n","responsetime":4,"server":"i-1e3ffbae","status":"OK","tags":["pw-dev","pw","tomcat","auth-services","packetbeat"],"type":"http"} } I used following stacks before, the Kibana shows the packetbeat events welll. packetbeat : 1.0.0-beta2 Redis:2.8.9 Logstash:1.4.2 ES: 1.4.4 Kibana: 4.1.1 ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 9:57pm UTC](https://discuss.elastic.co/t/packetbeat-data-not-going-from-es-to-kibana/28757/9 "2017-07-05T21:57:13Z")