Filebeat v6 and elasticsearch v6

Elastic Stack Beats
1

I set up filebeat and elasticsearch on the same machine and till v5.6 all was working good.
Now in elasticsearch log i found the following error line each per event:

2017-11-16T12:45:06+01:00 WARN Can not index event (status=400): {"type":"mapper_parsing_exception","reason":"Failed to parse mapping [doc]: Mapping definition for [error] has unsupported parameters: [properties : {code={type=long}, type={ignore_above=1024, type=keyword}, message={norms=false, type=text}}]","caused_by":{"type":"mapper_parsing_exception","reason":"Mapping definition for [error] has unsupported parameters: [properties : {code={type=long}, type={ignore_above=1024, type=keyword}, message={norms=false, type=text}}]"}}

What can it be the problem?

2

Can you share your filebeat configuration? The default index names in recent filebeat are versioned as well. There shouldn't be a conflict, unless you still write in the old index.

3

Also, using Kibana Console, can you do GET /filebeat-*/_mapping and GET _template/filebeat-* and paste the output in a pastebin or similar. Unless you have added custom fields to the templates, those two commands shouldn't leak any private information.

4

Here is the filebeat configuration

https://pastebin.com/embed_iframe/sXg1yU18

Thanks in advance

5

Note that logstash and winlogbeat works perfectly, only filebeat have problem.
Here is :
GET /filebeat-/_mapping
{
"error": {
"root_cause": [
{
"type": "index_not_found_exception",
"reason": "no such index",
"index_uuid": "na",
"index": "filebeat-"
}
],
"type": "index_not_found_exception",
"reason": "no such index",
"index_uuid": "na",
"index": "filebeat-*"
},
"status": 404
}

and

GET _template/filebeat-*
https://pastebin.com/miak7wbu

6

Hi,

Just had the same issue, you need to update the template for filebeat & es6.

You have the new mapping file in /etc/filebeat/filebeat.template-es6.json

I deleted the previous one and load the new one but there may be a better option if you need to keep your data.

curl -XDELETE http://localhost:9200/_template/filebeat
curl -XPUT -H 'Content-Type: application/json' http://localhost:9200/_template/filebeat -d@filebeat.template-es6x.json

See https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-template.html

Maybe there is a smarter way with some alias ?

1 Like
7

Seems a better option is to have a dedicated template for filebeat 6.0.0 to avoid compatibility issues between 2 différent versions.

And then voilà :slight_smile:

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.