Creating custom fields and populating them from log data

HI,
Can somebody help me in creating and populating the custom fields from log data which is being ingested in to ES using file beats?
I have gone through some of the documentation which discusses about Fields and Fields_under_root at https://www.elastic.co/guide/en/beats/filebeat/1.2/configuration-filebeat-options.html, This doesn’t show any example for extracting the data from log record to certain fields.

Kindly point me to an example with a regex pattern syntax. Thanks.

Sam

Filebeat doesn't support actually parsing the log files, because it's just a log shipper. I recommend using Logstash for that, see the Grok plugin.

Thank you!

Can you share some reference document with some examples please for custom fields creation in Logstash and grok patterns?

Thanks again.

Sam

One more question, can we use Filebeats for sending data to Logstash and parse the log there? Thanks.

Yes, that is one way it can be used. Checkout the Getting Started guide.

Once you have the data going to Logstash you would add a grok filter: See docs at Grok filter plugin | Logstash Reference [8.11] | Elastic

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.