Newly installed metricbeat=>logstash 6.0.0 Error (BeatsParser InvalidFrameProtocolException: Invalid Frame Type)


#1

When connecting metricbeats to logstash, I'm metricbeats is unable to connect:
"Caused by: org.logstash.beats.BeatsParser$InvalidFrameProtocolException: Invalid Frame Type, received"

We have EL5.6+Xpack , Kib5.6 +Xpack (NonSSL passwords for EL5.6)

  • I installed Metricbeat 6.1.1 (with a password and hooked it into ES with no problems.

  • I installed Logstash 5.x and upgraded it to EL 6.x (with a username + password), and it connected to EL fine. (using http inputs). I updated the logstash-input-beats.

  • When I change Logstash to "input { beats { port => "5044" } }". And repointed MetricBeat (from EL to Logstash (with and without the password in the now-defunct EL stanza), I get:
    "Caused by: org.logstash.beats.BeatsParser$InvalidFrameProtocolException: Invalid Frame Type, received"

From MetricBeat I get a connection reset by peer.

===
Metric Beat config
egrep -v "^[ ]#|^$" /etc/metricbeat/metricbeat.yml
metricbeat.config.modules:
path: ${path.config}/modules.d/
.yml
reload.enabled: true
reload.period: 10s
setup.template.settings:
index.number_of_shards: 1
index.codec: best_compression
setup.kibana:
host: "localhost:5601"
output.elasticsearch:
hosts: ["localhost:5044", "10.x.y.z:5044"]
logging.level: debug

========

input {
beats {
port => "5044"
}
}
filter {
grok {
match => {"message" => [ "hdfs%{GREEDYDATA:datanode_process}org.apache.hadoop.hdfs.server.datanode.DataNode",
"##REAL##%{GREEDYDATA:real_message}",
"%{GREEDYDATA:basic_message}"]}
}
if [datanode_process] {
mutate {
replace => { "message" => "%{datanode_process}" }
add_tag => ["DataNode"]
}
} else if [real_message] {
mutate {
replace => { "message" => "%{real_message}" }
add_tag => ["Real"]
}
} else {
mutate {
add_tag => ["Basic"]
}
}
}
output {
elasticsearch {
hosts => ["xxx:9200", "yyy:9200", "zzz:9200"]
user => username
password => pass
manage_template => false
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
}


(Andrew Kroh) #2

Please share your Metricbeat config.


#3

Added.


(Andrew Kroh) #4

Try using the Logstash output. https://www.elastic.co/guide/en/beats/metricbeat/current/logstash-output.html


#5

Thank you. Forgot to change the comments in the config.


(system) #6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.