# After insert document\_id at logstash, elastic stops receiveing data **URL:** \ **Category:** Elasticsearch\ **Created:** [September 20, 2017, 6:08pm UTC](https://discuss.elastic.co/t/after-insert-document-id-at-logstash-elastic-stops-receiveing-data/101223 "2017-09-20T18:08:27Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![paulo\_bruck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/paulo_bruck/32/22285_2.png) [@paulo\_bruck](https://discuss.elastic.co/u/paulo_bruck)\ **Post date:** [September 20, 2017, 6:08pm UTC](https://discuss.elastic.co/t/after-insert-document-id-at-logstash-elastic-stops-receiveing-data/101223/1 "2017-09-20T18:08:27Z")
HI guys Using debian + logstash+kibana+logstash version 5.6.0 If I insert document\_id at output elastic , I do not receive any new input at elasticsearch. Part of logstash file: input { file { path =\> "/var/log/firewall/firewall.log" type =\> "firewall" } } .... output { elasticsearch { hosts =\> ["127.0.0.1:9200"] index =\> "logstash-firewall-%{+YYYY.MM.dd}" **document\_id =\> "%{type}"** } } and letting logs at debug in logstash.yml I can see logs whithout error... What is wrong? I have already try to clean all indices and restart elasticsearch but no difference.....8( If I remove document\_id from logstash and restart it all goes right... part og logstash logs: [2017-09-20T11:18:10,674][DEBUG][logstash.instrument.periodicpoller.cgroup] Error, cannot retrieve cgroups i nformation {:exception=\>"Errno::ENOENT", :message=\>"No such file or directory - sys/fs/cgroup/cpuacct/cpu.cfs\_period\_us"} [2017-09-20T11:18:11,094][DEBUG][logstash.pipeline] Pushing flush onto pipeline [2017-09-20T11:18:14,037][DEBUG][logstash.inputs.file] each: file grew: /var/log/firewall/firewall.log: old size 1618301, new size 1618595 [2017-09-20T11:18:14,038][DEBUG][logstash.inputs.file] Received line {:path=\>"/var/log/firewall/firewal l.log", :text=\>"Sep 20 11:18:13 zeus kernel: [72287.762928] fir:block\_input\_tcp\_wan2 IN=wan2 OUT= PHYSIN=enp6s1 MAC=00:e0:4c:51:01:66:00:01:5c:80:60:46:08:00 SRC=78.120.52.31 DST=201.6.110.223 LEN=60 TOS=0x00 PREC=0x00 TTL=44 ID=39664 DF PROTO=TCP SPT=42003 DPT=51413 WINDOW=7300 RES=0x00 SYN URGP=0 MARK=0x2 "} [2017-09-20T11:18:14,048][DEBUG][logstash.pipeline] filter received {"event"=\>{"@version"=\>"1", "hos t"=\>"zeus", "path"=\>"/var/log/firewall/firewall.log", "@timestamp"=\>2017-09-20T14:18:14.038Z, "message"=\>"Se p 20 11:18:13 zeus kernel: [72287.762928] fir:block\_input\_tcp\_wan2 IN=wan2 OUT= PHYSIN=enp6s1 MAC=00:e0:4c:5 1:01:66:00:01:5c:80:60:46:08:00 SRC=78.120.52.31 DST=201.6.110.223 LEN=60 TOS=0x00 PREC=0x00 TTL=44 ID=39664 DF PROTO=TCP SPT=42003 DPT=51413 WINDOW=7300 RES=0x00 SYN URGP=0 MARK=0x2 ", "type"=\>"firewall"}} [2017-09-20T11:18:14,048][DEBUG][logstash.filters.grok] Running grok filter {:event=\>2017-09-20T14:18:14 .038Z zeus Sep 20 11:18:13 zeus kernel: [72287.762928] fir:block\_input\_tcp\_wan2 IN=wan2 OUT= PHYSIN=enp6s1 MAC=00:e0:4c:51:01:66:00:01:5c:80:60:46:08:00 SRC=78.120.52.31 DST=201.6.110.223 LEN=60 TOS=0x00 PREC=0x00 TTL=44 ID=39664 DF PROTO=TCP SPT=42003 DPT=51413 WINDOW=7300 RES=0x00 SYN URGP=0 MARK=0x2 } [2017-09-20T11:18:14,049][DEBUG][logstash.filters.grok] Event now: {:event=\>2017-09-20T14:18:14.038Z ze us Sep 20 11:18:13 zeus kernel: [72287.762928] fir:block\_input\_tcp\_wan2 IN=wan2 OUT= PHYSIN=enp6s1 MAC=00:e0:4c:51:01:66:00:01:5c:80:60:46:08:00 SRC=78.120.52.31 DST=201.6.110.223 LEN=60 TOS=0x00 PREC=0x00 TTL=44 ID=39664 DF PROTO=TCP SPT=42003 DPT=51413 WINDOW=7300 RES=0x00 SYN URGP=0 MARK=0x2 } Any ideas? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [September 20, 2017, 6:13pm UTC](https://discuss.elastic.co/t/after-insert-document-id-at-logstash-elastic-stops-receiveing-data/101223/2 "2017-09-20T18:13:39Z")
> [@paulo\_bruck](#): > > document\_id =\> "%{type}" This means that all documents will get exactly the same ID, resulting in the same document being overwritten repeatedly. This will be very slow, but also looks incorrect to me - what is it you are looking to achieve with this setting? ---
**Author:** ![paulo\_bruck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/paulo_bruck/32/22285_2.png) [@paulo\_bruck](https://discuss.elastic.co/u/paulo_bruck)\ **Post date:** [September 20, 2017, 6:25pm UTC](https://discuss.elastic.co/t/after-insert-document-id-at-logstash-elastic-stops-receiveing-data/101223/3 "2017-09-20T18:25:36Z")
Hi Christian The main reason is that comparing to metricbeat, I supose that is the way I can see at kibana an easy url. I would like to see a dasboard like below: [http://xxx:5601/app/kibana#/dashboard/firewall?\_g=()](http://xxx:5601/app/kibana#/dashboard/firewall?_g=()) Is that correct ? or I am using a wrong way? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [September 20, 2017, 6:27pm UTC](https://discuss.elastic.co/t/after-insert-document-id-at-logstash-elastic-stops-receiveing-data/101223/4 "2017-09-20T18:27:16Z")
Leave out the document\_id parameter and Elasticsearch will generate an id for each document. ---
**Author:** ![paulo\_bruck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/paulo_bruck/32/22285_2.png) [@paulo\_bruck](https://discuss.elastic.co/u/paulo_bruck)\ **Post date:** [September 20, 2017, 6:30pm UTC](https://discuss.elastic.co/t/after-insert-document-id-at-logstash-elastic-stops-receiveing-data/101223/5 "2017-09-20T18:30:11Z")
ok thanks Christian . Do you know how can I construct a dashboard with name instead indice? or should I ask at kibana forum? Something like [http://xxx:5601/app/kibana#/dashboard/](http://xxx:5601/app/kibana#/dashboard/) **firewall**?\_g=() ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [October 18, 2017, 6:30pm UTC](https://discuss.elastic.co/t/after-insert-document-id-at-logstash-elastic-stops-receiveing-data/101223/6 "2017-10-18T18:30:43Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.