Creating custom fields and populating them from log data

Elastic Stack Beats
1

HI,
Can somebody help me in creating and populating the custom fields from log data which is being ingested in to ES using file beats?
I have gone through some of the documentation which discusses about Fields and Fields_under_root at https://www.elastic.co/guide/en/beats/filebeat/1.2/configuration-filebeat-options.html, This doesn’t show any example for extracting the data from log record to certain fields.

Kindly point me to an example with a regex pattern syntax. Thanks.

Sam

2

Filebeat doesn't support actually parsing the log files, because it's just a log shipper. I recommend using Logstash for that, see the Grok plugin.

3

Thank you!

Can you share some reference document with some examples please for custom fields creation in Logstash and grok patterns?

Thanks again.

Sam

4

One more question, can we use Filebeats for sending data to Logstash and parse the log there? Thanks.

5

Yes, that is one way it can be used. Checkout the Getting Started guide.

Once you have the data going to Logstash you would add a grok filter: See docs at Grok filter plugin | Logstash Reference [8.11] | Elastic

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.