Updated filebeat (from 6.2.4 to 7.6 ) drops all log.*

[mishat-realityi (Misha)]

Hi everyone,
I have a stack filebeat > logstash > elasticsearch
Here is my config of filebeat.

    filebeat.inputs:
    - type: log
     enabled: true
     paths:
       - C:\folder\*
    json.keys_under_root: true
    output.logstash:
       hosts: [my.server:5044]

After migrating to filebeat 7.6.4 my log.message stop arriving to elasticsearch but everything else still does. It was obvious that I got a conflict with new ECS.

I went through migration documentation but still can find a proper solution to my problem. I don't use filebeat modules and changing a current log schema aren't an option. I've also tried different processors but no luck either.

Would love some thoughts on how to resolve this.
Thank you,

Misha

[mishat-realityi (Misha)]

Will the best solution be to create a new filebeat module ?

[webmat (Mathieu Martin)]

The existing documentation focuses on data sources provided by Elastic indeed. Most of those have been migrated to ECS.

However both Filebeat and Logstash can send events to custom indices per data source. So if you create an index template that defines the schema as you had it in 6.x, then point the relevant stream of events to an index that matches this index template, you should be good to continue capturing your events exactly as you had them in 6.x.