We've recently upgraded our elastic stack and have upgraded our Filebeat version from 5.6.14 to 7.8.1. The old version used to provide a "source" field that contained the path name of the log file. This field is missing in the new version and i don't see any field in the output that has the log file path name in it. Is there some way I can configure Filebeat 7.8.1 to provide this field. I can't find this in the documentation. Thanks.
@PMDubuc A bit unsure which one you mean, if you mean the specific logfile that is currently being processed by filebeat, it should be under log.* fields.
Do you use this with modules, or is this a custom one using filebeat.yml?
Do you have any custom input processors or ingest pipelines that might remove fields?
I'm not using modules. I use inputs instead. I don't see the log.* fields. I think I know what's happening. The log.* fields are getting overwritten by a log field in log messages that are formatted in JSON. Is there any way in Filebeat to rename the log.* fields to something like filebeat-log.* so they aren't overwritten when these JSON logs are parsed in the Logstash filter?
I solved this by renaming the log field at the top of my logstash filter:
mutate {
rename => { "log" => "filebeat-log"}
}
so the log.* fields become filebeat-log.* and they don't get overwritten by messages with that field name in them.
Happy to hear you got it resolved! 
Another option if you like to resolve it on the filebeat side is to use processors, we have one for renaming for example.
Thanks! Since we have fewer instances of Logstash than Filebeat this way will be easier for us.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.