Hello. I am running Filebeat 6.4 and Elasticsearch 6.3 on Amazon Linux 2018.03 and am receiving the following error in my filebeat log:
{
"level":"error",
"timestamp":"2018-11-16T03:56:40.395Z",
"caller":"pipeline/output.go:100",
"message":"Failed to connect to backoff(elasticsearch(hostname)):
Connection marked as failed because the onConnect callback failed:
Error loading pipeline for fileset system/auth:
couldn't load pipeline:
couldn't load json. Error: 500 Server Error: {
\"error\":{
\"root_cause\":[
{\"type\":\"not_x_content_exception\",
\"reason\":\"Compressor detection can only be called on some xcontent bytes or compressed xcontent bytes\"}
],
\"type\":\"not_x_content_exception\",
\"reason\":\"Compressor detection can only be called on some xcontent bytes or compressed xcontent bytes\"},
\"status\":500}.
Response body: {\"error\":{\"root_cause\":[{\"type\":\"not_x_content_exception\",\"reason\":\"Compressor detection can only be called on some xcontent bytes or compressed xcontent bytes\"}],\"type\":\"not_x_content_exception\",\"reason\":\"Compressor detection can only be called on some xcontent bytes or compressed xcontent bytes\"},\"status\":500}"
}
My filebeat config is (I tested with this file, nothing has been removed after the fact except hostname and some root fields):
filebeat.modules:
- module: system
syslog:
enabled: true
auth:
enabled: true
logging:
level: info
json: true
files:
path: /var/log/filebeat
keepfiles: 7
fields_under_root: true
fields:
"@instance_id": x
"@server_type": y
setup:
template:
name: filebeat
pattern: filebeat-*
output.elasticsearch:
hosts: [...]
compression_level: 6
indices:
- index: filebeat-%{[beat.version]}-%{+yyyy.MM.dd}-unknown
Does anyone know why this happens? How can I fix it?
EDIT: I upgraded to Filebeat 6.5.0 and no change