Filebeat field depending on Path

[OptiFine (Arthur Jamain)]

Hi,

I am trying to add a specific field which will be varying depending on path.

I have logs coming from 6 different environment respectively 'prod', '101', '102', '201', '202', '301'. It is planned to add an other one soon so I would like to do something abstract.

I would like to generate automatically a field by extracting the information from the path.

Ex:

paths : /var/log/prod/App/app.log

And my fields would be :

fields: { env: prod}

I don't know how I can do that and if it is possible.

Some help ? Thanks in advance.

[pierhugues (Pier-Hugues Pellerin)]

Hello @OptiFine,

It's not currently possible in Filebeat to do that; we do have the internal discussion to add a way to extract information or fields to events but no clear short-term plan.

But, you could send your events to an ingest pipeline and use the grok processor to extract that information? Each event has the original file, so you use that as the target field.

Thanks

[OptiFine (Arthur Jamain)]

Hi Pierhugues,

You're right it might be a good idea to use a grok filter in my logstash... I will see if the idea is approved by the team !

Thanks for your answer !

[pierhugues (Pier-Hugues Pellerin)]

Yes logstash would also work!