After inserting document_id at logstash, elastic stops receiveing data

Elastic Stack Kibana
1

HI guys

Using debian + logstash+kibana+logstash version 5.6.0

If I insert document_id at output elastic , I do not receive any new input at lelasticsearch.

Part of logstash file:

input {
file {
path => "/var/log/firewall/firewall.log"
type => "firewall"
}
}
....

output {

stdout { codec => rubydebug }

    elasticsearch {
            hosts => ["127.0.0.1:9200"]
            index => "logstash-firewall-%{+YYYY.MM.dd}"
            **document_id => "%{type}"**
    }

}

and letting logs at debug in logstash.yml I can see logs whithout error... What is wrong?
I have already try to clean all indices and restart elasticsearch but no difference.....8(

If I remove document_id from logstash and restart it all go right...

part og logstash logs:

[2017-09-20T11:18:10,674][DEBUG][logstash.instrument.periodicpoller.cgroup] Error, cannot retrieve cgroups i
nformation {:exception=>"Errno::ENOENT", :message=>"No such file or directory - /sys/fs/cgroup/cpuacct/cpu.c
fs_period_us"}
[2017-09-20T11:18:11,094][DEBUG][logstash.pipeline ] Pushing flush onto pipeline
[2017-09-20T11:18:14,037][DEBUG][logstash.inputs.file ] each: file grew: /var/log/firewall/firewall.log:
old size 1618301, new size 1618595
[2017-09-20T11:18:14,038][DEBUG][logstash.inputs.file ] Received line {:path=>"/var/log/firewall/firewal
l.log", :text=>"Sep 20 11:18:13 zeus kernel: [72287.762928] fir:block_input_tcp_wan2 IN=wan2 OUT= PHYSIN=enp
6s1 MAC=00:e0:4c:51:01:66:00:01:5c:80:60:46:08:00 SRC=78.120.52.31 DST=201.6.110.223 LEN=60 TOS=0x00 PREC=0x
00 TTL=44 ID=39664 DF PROTO=TCP SPT=42003 DPT=51413 WINDOW=7300 RES=0x00 SYN URGP=0 MARK=0x2 "}
[2017-09-20T11:18:14,048][DEBUG][logstash.pipeline ] filter received {"event"=>{"@version"=>"1", "hos
t"=>"zeus", "path"=>"/var/log/firewall/firewall.log", "@timestamp"=>2017-09-20T14:18:14.038Z, "message"=>"Se
p 20 11:18:13 zeus kernel: [72287.762928] fir:block_input_tcp_wan2 IN=wan2 OUT= PHYSIN=enp6s1 MAC=00:e0:4c:5
1:01:66:00:01:5c:80:60:46:08:00 SRC=78.120.52.31 DST=201.6.110.223 LEN=60 TOS=0x00 PREC=0x00 TTL=44 ID=39664
DF PROTO=TCP SPT=42003 DPT=51413 WINDOW=7300 RES=0x00 SYN URGP=0 MARK=0x2 ", "type"=>"firewall"}}
[2017-09-20T11:18:14,048][DEBUG][logstash.filters.grok ] Running grok filter {:event=>2017-09-20T14:18:14
.038Z zeus Sep 20 11:18:13 zeus kernel: [72287.762928] fir:block_input_tcp_wan2 IN=wan2 OUT= PHYSIN=enp6s1 M
AC=00:e0:4c:51:01:66:00:01:5c:80:60:46:08:00 SRC=78.120.52.31 DST=201.6.110.223 LEN=60 TOS=0x00 PREC=0x00 TT
L=44 ID=39664 DF PROTO=TCP SPT=42003 DPT=51413 WINDOW=7300 RES=0x00 SYN URGP=0 MARK=0x2 }
[2017-09-20T11:18:14,049][DEBUG][logstash.filters.grok ] Event now: {:event=>2017-09-20T14:18:14.038Z ze
us Sep 20 11:18:13 zeus kernel: [72287.762928] fir:block_input_tcp_wan2 IN=wan2 OUT= PHYSIN=enp6s1 MAC=00:e0
:4c:51:01:66:00:01:5c:80:60:46:08:00 SRC=78.120.52.31 DST=201.6.110.223 LEN=60 TOS=0x00 PREC=0x00 TTL=44 ID=
39664 DF PROTO=TCP SPT=42003 DPT=51413 WINDOW=7300 RES=0x00 SYN URGP=0 MARK=0x2 }

Any ideas? Please let me know if it is the correct forum...80))

2

opps sorry guys , now I see that I posted at wrong forun...

3

No worries!

For anyone that sees this:

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.