Filebeat no more working - Can not index event (status=400)

[toasti (Bastian)]

Hello there,

Since today at 2am my filebat was working well, now I got no more data and the following error messages:

2016-09-22T11:53:04+02:00 WARN Can not index event (status=400): "MapperParsingException[mapping [_default_]]; nested: MapperParsingException[No handler for type [keyword] declared on field [hostname]]; "
2016-09-22T11:53:26+02:00 INFO Non-zero metrics in the last 30s: libbeat.es.publish.read_bytes=873 registar.states.total=2 filebeat.harvester.started=1 libbeat.es.publish.write_bytes=1411     

libbeat.es.call_count.PublishEvents=1 filebeat.harvester.running=1 publish.events=2 registrar.state_updates=2 libbeat.es.published_and_acked_events=1 libbeat.publisher.published_events=1 filebeat.harvester.open_files=1

Don't know what happen - do you have an idea?

Thank you!

[ruflin]

It seems like the mapping on your elasticsearch instance changed. Do you have the filebeat index template loaded on your elasticsearch instance?

[toasti (Bastian)]

Sorry, but I don't understand what do you mean exactly. I am using it since Monday and for me it's still very complex.

Do you mean "filebeat.template.json" and "filebeat.template-es2x.json"?

What can I do or check to find the failure?

[ruflin]

Did you follow the getting started guide and applied this step here? https://www.elastic.co/guide/en/beats/filebeat/1.3/filebeat-template.html Which version of filebeat are you using?

[toasti (Bastian)]

Yes, I did and it already worked.

I am using version filebeat-5.0.0-alpha5-windows-x86_64.

Do you mean uninstall and reinstall works? In the debug log I can see that filebeat send logs, but afterwards the error message above comes up.

I tried also to delete the index of filebeat and create it again. No change, unable to fetch mapping.

[toasti (Bastian)]

I solved it, hope that is correct now:

Based on this post from Andrew I did the same with filebeat, also using the filebeat.yml from version 1.3.1:

Issue shipping Winlogbeat 5.x to Elasticsearch 2.x

It looks like you are using the Winlogbeat 5.0.0-alpha1 with Elasticsearch 1.x or 2.x? There is an incompatibility with the winlogbeat.template.json index template provided in Winlogbeat v5 and earlier versions of Elasticsearch because the template uses the text keyword that was introduced in ES v5. To continue to use Winlogbeat 5.x with ES 1.x or 2.x you'll need to grab the index template provided in Winlogbeat 1.x and install it to ES. Cleanup: Stop Winlogbeat. Delete the .winlogbeat.yml r…

[ruflin]

Filebeat 5.x automatically installs the template and picks the right one for the elasticsearch version. I think what solved the problem is that you removed the index template (not only the index).

Thanks for sharing your solution.

[toasti (Bastian)]

I did a test and installed the 5.0 template again - not working.
Copied back the 1.3.1 template - working.

Is my Elasticsearch version not the right one for 5.0? It's 1.7.3.

[ruflin]

For 1.x and 2.x of elasticsearch, you need the 2.x template. Only for 5.x the 5.0 template is needed.