Creating custom fields and populating them from log data


(sam) #1

HI,
Can somebody help me in creating and populating the custom fields from log data which is being ingested in to ES using file beats?
I have gone through some of the documentation which discusses about Fields and Fields_under_root at https://www.elastic.co/guide/en/beats/filebeat/1.2/configuration-filebeat-options.html, This doesn’t show any example for extracting the data from log record to certain fields.

Kindly point me to an example with a regex pattern syntax. Thanks.

Sam


(Tudor Golubenco) #2

Filebeat doesn't support actually parsing the log files, because it's just a log shipper. I recommend using Logstash for that, see the Grok plugin.


(sam) #3

Thank you!

Can you share some reference document with some examples please for custom fields creation in Logstash and grok patterns?

Thanks again.

Sam


(sam) #4

One more question, can we use Filebeats for sending data to Logstash and parse the log there? Thanks.


(Andrew Kroh) #5

Yes, that is one way it can be used. Checkout the Getting Started guide.

Once you have the data going to Logstash you would add a grok filter: See docs at https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html


(system) #6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.