Hey folks,
after a long time away from the elk stack I wanted to set up another testing scenario. I followed the latest stack guide from here: https://www.elastic.co/guide/en/elastic-stack-get-started/current/get-started-elastic-stack.html.
My PROBLEM:
connecting beats <-> logstash. The error msg is following beneath of the description.
What is working: Metric-/Filebeat to Kibana/Elasticsearch
What doesn't work: Filebeat via Logstash to Elesticsearch/Kibana
Without logstash everything works like a charm. I followed and crosschecked everything twice. I even tried a third party guide from digital ocean: https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-elastic-stack-on-ubuntu-16-04 on a clean vm.
In case i missed to provide some informations I'll deliver them as fast as possible.
ELK
All my elk components are in version 6.5.4 and installed from repo
debian:
Linux elk-stack2 4.9.0-6-amd64 #1 SMP Debian 4.9.82-1+deb9u3 (2018-03-02) x86_64 GNU/Linux
java -version
openjdk version "1.8.0_181"
OpenJDK Runtime Environment (build 1.8.0_181-8u181-b13-2~deb9u1-b13)
OpenJDK 64-Bit Server VM (build 25.181-b13, mixed mode)
filebeat.yml
#================================ Outputs =====================================
# Configure what output to use when sending the data collected by the beat.
#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
# Array of hosts to connect to.
#hosts: ["localhost:9200"]
# Optional protocol and basic auth credentials.
#protocol: "https"
#username: "elastic"
#password: "changeme"
#----------------------------- Logstash output --------------------------------
#output.logstash:
# The Logstash hosts
hosts: ["localhost:5044"]
# Optional SSL. By default is off.
# List of root certificates for HTTPS server verifications
#ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]
# Certificate for SSL client authentication
#ssl.certificate: "/etc/pki/client/cert.pem"
# Client Certificate Key
#ssl.key: "/etc/pki/client/cert.key"
Logstash Pipeline:
input {
beats {
port => 5044
}
}
# The filter part of this file is commented out to indicate that it
# is optional.
# filter {
#
# }
output {
elasticsearch {
hosts => "localhost:9200"
manage_template => false
index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
}
}
Logstash Error log:
Jan 20 20:54:12 elk-stack2 logstash[1687]: [2019-01-20T20:54:12,444][INFO ][org.logstash.beats.BeatsHandler] [local: 0:0:0:0:0:0:0:1:5044, remote: 0:0:0:0:0:0:0:1:60972] Handling exception: org.logstash.beats.BeatsParser$InvalidFrameProtocolException: Invalid Frame Type, received: 84
Jan 20 20:54:12 elk-stack2 logstash[1687]: [2019-01-20T20:54:12,444][WARN ][io.netty.channel.DefaultChannelPipeline] An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
Jan 20 20:54:12 elk-stack2 logstash[1687]: io.netty.handler.codec.DecoderException: org.logstash.beats.BeatsParser$InvalidFrameProtocolException: Invalid Frame Type, received: 84
Jan 20 20:54:12 elk-stack2 logstash[1687]: at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:459) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]
Jan 20 20:54:12 elk-stack2 logstash[1687]: at io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:392) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]
Jan 20 20:54:12 elk-stack2 logstash[1687]: at io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:359) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]
Jan 20 20:54:12 elk-stack2 logstash[1687]: at io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:342) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]
Jan 20 20:54:12 elk-stack2 logstash[1687]: at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]
Jan 20 20:54:12 elk-stack2 logstash[1687]: at io.netty.channel.AbstractChannelHandlerContext.access$300(AbstractChannelHandlerContext.java:38) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]
Jan 20 20:54:12 elk-stack2 logstash[1687]: at io.netty.channel.AbstractChannelHandlerContext$4.run(AbstractChannelHandlerContext.java:236) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]
Jan 20 20:54:12 elk-stack2 logstash[1687]: at io.netty.util.concurrent.DefaultEventExecutor.run(DefaultEventExecutor.java:66) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]
Jan 20 20:54:12 elk-stack2 logstash[1687]: at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) [netty-all-4.1.18.Final.jar:4.1.18.Final]
Jan 20 20:54:12 elk-stack2 logstash[1687]: at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-all-4.1.18.Final.jar:4.1.18.Final]
Jan 20 20:54:12 elk-stack2 logstash[1687]: at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]
Jan 20 20:54:12 elk-stack2 logstash[1687]: Caused by: org.logstash.beats.BeatsParser$InvalidFrameProtocolException: Invalid Frame Type, received: 84
Jan 20 20:54:12 elk-stack2 logstash[1687]: at org.logstash.beats.BeatsParser.decode(BeatsParser.java:92) ~[logstash-input-beats-5.1.6.jar:?]
Jan 20 20:54:12 elk-stack2 logstash[1687]: at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]
Jan 20 20:54:12 elk-stack2 logstash[1687]: at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]
Jan 20 20:54:12 elk-stack2 logstash[1687]: ... 10 more