Hello, I have an issue with logstash:
Versions: elasticsearch-6.4.0-1.noarch, kibana-6.4.0-1.x86_64, logstash-6.4.0-1.noarch
The log:
` < [2018-09-11T11:22:42,976][INFO ][logstash.pipeline ] Pipeline has terminated {:pipeline_id=>"main", :thread=>"#<Thread:0x6d8f95d2 run>"}
[2018-09-11T14:02:59,840][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"6.4.0"}
[2018-09-11T14:03:05,682][INFO ][logstash.pipeline ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[2018-09-11T14:03:06,857][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>, :added=>[http://10.60.192.52:9200/]}}
[2018-09-11T14:03:06,961][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://10.60.192.52:9200/, :path=
"/"}
[2018-09-11T14:03:07,387][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"http://10.60.192.52:9200/"}
[2018-09-11T14:03:07,551][INFO ][logstash.outputs.elasticsearch] ES Output version determined {:es_version=>6}
[2018-09-11T14:03:07,575][WARN ][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: thetypeevent field won't be used to determine the document _type {:es_version=>6}
[2018-09-11T14:03:07,624][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::Elasticsearch", :hosts=>["http://10.60.192.52:9200"]}
[2018-09-11T14:03:07,684][INFO ][logstash.outputs.elasticsearch] Using mapping template from {:path=>nil}
[2018-09-11T14:03:07,773][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"template"=>"logstash-", "version"=>60001, "settings"=>{"index.refresh
_interval"=>"5s"}, "mappings"=>{"default"=>{"dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}
}}, {"string_fields"=>{"match"=>"", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "prop
erties"=>{"@timestamp"=>{"type"=>"date"}, "@version"=>{"type"=>"keyword"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{
"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}}
[2018-09-11T14:03:09,115][INFO ][logstash.inputs.beats ] Beats inputs: Starting input listener {:address=>"0.0.0.0:5044"}
[2018-09-11T14:03:09,155][INFO ][logstash.pipeline ] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0x5979a392 run>"}
[2018-09-11T14:03:09,361][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>}
[2018-09-11T14:03:09,657][INFO ][org.logstash.beats.Server] Starting server on port: 5044
[2018-09-11T14:03:10,252][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
[2018-09-11T14:03:45,572][INFO ][org.logstash.beats.BeatsHandler] [local: 10.60.192.52:5044, remote: 10.60.194.36:58304] Handling exception: org.logstash.beats.BeatsParser$InvalidFramePro
tocolException: Invalid Frame Type, received: 3
[2018-09-11T14:03:45,575][WARN ][io.netty.channel.DefaultChannelPipeline] An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handl
er in the pipeline did not handle the exception.
io.netty.handler.codec.DecoderException: org.logstash.beats.BeatsParser$InvalidFrameProtocolException: Invalid Frame Type, received: 3
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:459) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]
at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:38) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]
at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:353) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]
at io.netty.util.concurrent.DefaultEventExecutor.run(DefaultEventExecutor.java:66) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) [netty-all-4.1.18.Final.jar:4.1.18.Final]
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-all-4.1.18.Final.jar:4.1.18.Final]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]
Caused by: org.logstash.beats.BeatsParser$InvalidFrameProtocolException: Invalid Frame Type, received: 3
at org.logstash.beats.BeatsParser.decode(BeatsParser.java:92) ~[logstash-input-beats-5.1.6.jar:?]
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]
... 8 more /> `
The conf file:
[root@elk_server ~]# cat /etc/logstash/logstash.yml |grep -v '#'
path.data: /elk/logstash
path.logs: /var/log/logstash
[root@elk_server ~]# cat /etc/logstash/conf.d/*.conf
filter {
if [type] == "syslog" {
grok {
match => { "message" => "%{SYSLOGLINE}" }
}
date {
match => [ "timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
}
input {
beats {
port => 5044
}
}
output {
elasticsearch {
hosts => ["http://10.60.192.52:9200"]
index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
}
}
The filebeat conf on client:
root@
# cat filebeat.yml |grep -v '#'
filebeat:
prospectors:
-
paths:
- /var/log/.log
- /var/log/zabbix/.log
- /var/log/syslog
- /var/log/postgres/.log
- /opt/apache-tomcat-7.0.82/logs/.log
- /opt/apache-tomcat-6.0.9/logs/.log
- /opt/alfresco/tomcat/logs/.log
input_type: log
Could you help me, please?