# After inserting document\_id at logstash, elastic stops receiveing data **URL:** \ **Category:** Kibana\ **Created:** [September 20, 2017, 2:33pm UTC](https://discuss.elastic.co/t/after-inserting-document-id-at-logstash-elastic-stops-receiveing-data/101191 "2017-09-20T14:33:18Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![paulo\_bruck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/paulo_bruck/32/22285_2.png) [@paulo\_bruck](https://discuss.elastic.co/u/paulo_bruck)\ **Post date:** [September 20, 2017, 2:33pm UTC](https://discuss.elastic.co/t/after-inserting-document-id-at-logstash-elastic-stops-receiveing-data/101191/1 "2017-09-20T14:33:18Z")
HI guys Using debian + logstash+kibana+logstash version 5.6.0 If I insert document\_id at output elastic , I do not receive any new input at lelasticsearch. Part of logstash file: input { file { path =\> "/var/log/firewall/firewall.log" type =\> "firewall" } } .... output { # stdout { codec =\> rubydebug } ``` elasticsearch { hosts => ["127.0.0.1:9200"] index => "logstash-firewall-%{+YYYY.MM.dd}" **document_id => "%{type}"** } ``` } and letting logs at debug in logstash.yml I can see logs whithout error... What is wrong? I have already try to clean all indices and restart elasticsearch but no difference.....8( If I remove document\_id from logstash and restart it all go right... part og logstash logs: [2017-09-20T11:18:10,674][DEBUG][logstash.instrument.periodicpoller.cgroup] Error, cannot retrieve cgroups i nformation {:exception=\>"Errno::ENOENT", :message=\>"No such file or directory - /sys/fs/cgroup/cpuacct/cpu.c fs\_period\_us"} [2017-09-20T11:18:11,094][DEBUG][logstash.pipeline] Pushing flush onto pipeline [2017-09-20T11:18:14,037][DEBUG][logstash.inputs.file] each: file grew: /var/log/firewall/firewall.log: old size 1618301, new size 1618595 [2017-09-20T11:18:14,038][DEBUG][logstash.inputs.file] Received line {:path=\>"/var/log/firewall/firewal l.log", :text=\>"Sep 20 11:18:13 zeus kernel: [72287.762928] fir:block\_input\_tcp\_wan2 IN=wan2 OUT= PHYSIN=enp 6s1 MAC=00:e0:4c:51:01:66:00:01:5c:80:60:46:08:00 SRC=78.120.52.31 DST=201.6.110.223 LEN=60 TOS=0x00 PREC=0x 00 TTL=44 ID=39664 DF PROTO=TCP SPT=42003 DPT=51413 WINDOW=7300 RES=0x00 SYN URGP=0 MARK=0x2 "} [2017-09-20T11:18:14,048][DEBUG][logstash.pipeline] filter received {"event"=\>{"@version"=\>"1", "hos t"=\>"zeus", "path"=\>"/var/log/firewall/firewall.log", "@timestamp"=\>2017-09-20T14:18:14.038Z, "message"=\>"Se p 20 11:18:13 zeus kernel: [72287.762928] fir:block\_input\_tcp\_wan2 IN=wan2 OUT= PHYSIN=enp6s1 MAC=00:e0:4c:5 1:01:66:00:01:5c:80:60:46:08:00 SRC=78.120.52.31 DST=201.6.110.223 LEN=60 TOS=0x00 PREC=0x00 TTL=44 ID=39664 DF PROTO=TCP SPT=42003 DPT=51413 WINDOW=7300 RES=0x00 SYN URGP=0 MARK=0x2 ", "type"=\>"firewall"}} [2017-09-20T11:18:14,048][DEBUG][logstash.filters.grok] Running grok filter {:event=\>2017-09-20T14:18:14 .038Z zeus Sep 20 11:18:13 zeus kernel: [72287.762928] fir:block\_input\_tcp\_wan2 IN=wan2 OUT= PHYSIN=enp6s1 M AC=00:e0:4c:51:01:66:00:01:5c:80:60:46:08:00 SRC=78.120.52.31 DST=201.6.110.223 LEN=60 TOS=0x00 PREC=0x00 TT L=44 ID=39664 DF PROTO=TCP SPT=42003 DPT=51413 WINDOW=7300 RES=0x00 SYN URGP=0 MARK=0x2 } [2017-09-20T11:18:14,049][DEBUG][logstash.filters.grok] Event now: {:event=\>2017-09-20T14:18:14.038Z ze us Sep 20 11:18:13 zeus kernel: [72287.762928] fir:block\_input\_tcp\_wan2 IN=wan2 OUT= PHYSIN=enp6s1 MAC=00:e0 :4c:51:01:66:00:01:5c:80:60:46:08:00 SRC=78.120.52.31 DST=201.6.110.223 LEN=60 TOS=0x00 PREC=0x00 TTL=44 ID= 39664 DF PROTO=TCP SPT=42003 DPT=51413 WINDOW=7300 RES=0x00 SYN URGP=0 MARK=0x2 } Any ideas? Please let me know if it is the correct forum...80)) ---
**Author:** ![paulo\_bruck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/paulo_bruck/32/22285_2.png) [@paulo\_bruck](https://discuss.elastic.co/u/paulo_bruck)\ **Post date:** [September 20, 2017, 6:02pm UTC](https://discuss.elastic.co/t/after-inserting-document-id-at-logstash-elastic-stops-receiveing-data/101191/2 "2017-09-20T18:02:05Z")
> [@paulo\_bruck](#): > > HI guys > > Using debian + logstash+kibana+logstash version 5.6.0 > > If I insert document\_id at output elastic , I do not receive any new input at lelasticsearch. > > Part of logstash file: > > input { > > file { > > path =\> "/var/log/firewall/firewall.log" > > type =\> "firewall" > > } > > } > > .... > > output { > > stdout { codec =\> rubydebug } > elasticsearch { > hosts =\> ["127.0.0.1:9200"] > index =\> "logstash-firewall-%{+YYYY.MM.dd}" > **document\_id =\> "%{type}"** > } > > } > > and letting logs at debug in logstash.yml I can see logs whithout error... What is wrong? > > I have already try to clean all indices and restart elasticsearch but no difference.....8( > > If I remove document\_id from logstash and restart it all go right... > > part og logstash logs: > > [2017-09-20T11:18:10,674][DEBUG][logstash.instrument.periodicpoller.cgroup] Error, cannot retrieve cgroups i > > nformation {:exception=\>"Errno::ENOENT", :message=\>"No such file or directory - /sys/fs/cgroup/cpuacct/cpu.c > > fs\_period\_us"} > > [2017-09-20T11:18:11,094][DEBUG][logstash.pipeline] Pushing flush onto pipeline > > [2017-09-20T11:18:14,037][DEBUG][logstash.inputs.file] each: file grew: /var/log/firewall/firewall.log: > > old size 1618301, new size 1618595 > > [2017-09-20T11:18:14,038][DEBUG][logstash.inputs.file] Received line {:path=\>"/var/log/firewall/firewal > > l.log", :text=\>"Sep 20 11:18:13 zeus kernel: [72287.762928] fir:block\_input\_tcp\_wan2 IN=wan2 OUT= PHYSIN=enp > > 6s1 MAC=00:e0:4c:51:01:66:00:01:5c:80:60:46:08:00 SRC=78.120.52.31 DST=201.6.110.223 LEN=60 TOS=0x00 PREC=0x > > 00 TTL=44 ID=39664 DF PROTO=TCP SPT=42003 DPT=51413 WINDOW=7300 RES=0x00 SYN URGP=0 MARK=0x2 "} > > [2017-09-20T11:18:14,048][DEBUG][logstash.pipeline] filter received {"event"=\>{"@version"=\>"1", "hos > > t"=\>"zeus", "path"=\>"/var/log/firewall/firewall.log", "@timestamp"=\>2017-09-20T14:18:14.038Z, "message"=\>"Se > > p 20 11:18:13 zeus kernel: [72287.762928] fir:block\_input\_tcp\_wan2 IN=wan2 OUT= PHYSIN=enp6s1 MAC=00:e0:4c:5 > > 1:01:66:00:01:5c:80:60:46:08:00 SRC=78.120.52.31 DST=201.6.110.223 LEN=60 TOS=0x00 PREC=0x00 TTL=44 ID=39664 > > DF PROTO=TCP SPT=42003 DPT=51413 WINDOW=7300 RES=0x00 SYN URGP=0 MARK=0x2 ", "type"=\>"firewall"}} > > [2017-09-20T11:18:14,048][DEBUG][logstash.filters.grok] Running grok filter {:event=\>2017-09-20T14:18:14 > > .038Z zeus Sep 20 11:18:13 zeus kernel: [72287.762928] fir:block\_input\_tcp\_wan2 IN=wan2 OUT= PHYSIN=enp6s1 M > > AC=00:e0:4c:51:01:66:00:01:5c:80:60:46:08:00 SRC=78.120.52.31 DST=201.6.110.223 LEN=60 TOS=0x00 PREC=0x00 TT > > L=44 ID=39664 DF PROTO=TCP SPT=42003 DPT=51413 WINDOW=7300 RES=0x00 SYN URGP=0 MARK=0x2 } > > [2017-09-20T11:18:14,049][DEBUG][logstash.filters.grok] Event now: {:event=\>2017-09-20T14:18:14.038Z ze > > us Sep 20 11:18:13 zeus kernel: [72287.762928] fir:block\_input\_tcp\_wan2 IN=wan2 OUT= PHYSIN=enp6s1 MAC=00:e0 > > :4c:51:01:66:00:01:5c:80:60:46:08:00 SRC=78.120.52.31 DST=201.6.110.223 LEN=60 TOS=0x00 PREC=0x00 TTL=44 ID= > > 39664 DF PROTO=TCP SPT=42003 DPT=51413 WINDOW=7300 RES=0x00 SYN URGP=0 MARK=0x2 } > > Any ideas? Please let me know if it is the correct forum...80)) opps sorry guys , now I see that I posted at wrong forun... ---
**Author:** ![tsullivan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tsullivan/32/31077_2.png) [@tsullivan](https://discuss.elastic.co/u/tsullivan)\ **Post date:** [September 20, 2017, 8:00pm UTC](https://discuss.elastic.co/t/after-inserting-document-id-at-logstash-elastic-stops-receiveing-data/101191/3 "2017-09-20T20:00:17Z")
No worries! For anyone that sees this: > [@After insert document\_id at logstash, elastic stops receiveing data](https://discuss.elastic.co/t/after-insert-document-id-at-logstash-elastic-stops-receiveing-data/101223): > > HI guys Using debian + logstash+kibana+logstash version 5.6.0 If I insert document\_id at output elastic , I do not receive any new input at elasticsearch. Part of logstash file: input { file { path =\> "/var/log/firewall/firewall.log" type =\> "firewall" } } .... output { elasticsearch { hosts =\> ["127.0.0.1:9200"] index =\> "logstash-firewall-%{+YYYY.MM.dd}" document\_id =\> "%{type}" } } and letting logs at debug in logstash.yml I can see logs whithout error... What is wrong? I h… ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [October 18, 2017, 8:00pm UTC](https://discuss.elastic.co/t/after-inserting-document-id-at-logstash-elastic-stops-receiveing-data/101191/4 "2017-10-18T20:00:22Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.