Field Extraction/Parsing

Elastic Stack Beats
1

Hi everyone, I'm a new user of the ELK stack.

I'm monitoring a file and I would like to extract fields (hereinafter a small extraction):

<Item> <title> Vulnerability in Cisco Products (March 6, 2019) </ title> <Link> https://www.certnazionale.it/news/2019/03/07/vulnerabilita-in-prodotti-cisco-6-marzo-2019/ </ link> <pubDate> Thu, 07 Mar 2019 10:01:10 +0000 </ pubDate> <dc: creator> <! [CDATA [National CERT]]> </ dc: creator> <Category> <! [CDATA [Vulnerability]]> </ category> <Category> <! [CDATA [cisco]]> </ category> <guid isPermaLink = "false"> https://www.certnazionale.it/?post_type=news&#038;p=1946 </ guid> <description> <! [CDATA [Cisco has released several security updates on March 6, 2019 that address multiple vulnerabilities in different products.]]> </ description> </ Item>

I would like to extract the folowing fields (with relative values):

title=Vulnerability in Cisco Products (March 6, 2019)

pudDate=Thu, 07 Mar 2019 10:01:10 +0000

description=Cisco has released several security updates on March 6, 2019 that address multiple vulnerabilities in different products.

Can you explain me how to extract fields at index time with Filebeat or Logstash?

Thanks

2

Hello @elborni96, You cannot do that in Filebeat at the moment, you will need to use logstash and the logstash-filter-xml

3

Hi @pierhugues,

I need to understand how to parse / extract fields of events that come to me through the Filbeat, regardless of the type of file (XML, HTML, JSON, syslog ...).

Could you explain how to do it via LogStash?

Regards,
Mirko

4

(XML, HTML, JSON, syslog ...).

The big picture would be something like this.

After that you can what you want with the data of the event.

5

@pierhugues

I have defined Filebeat input that monitor my file (/home/root/test.log).

After i send monitored data to Logstash using the output function.

I have configured yet on Logstash the beats input, but i need to parse the content of the received events.

These events can be xml, syslog, json or other format; so i need to understand how to extract certain fields.

On Splunk Enterprise i can parse or extract fields using the following syntax:

EXTRACT-host_field = $regex_to_extract_host

How can do the same using parsing/extraction features of Logstash?

Can i create custom searchable fields valorized from the regex results?

Thanks,
Mirko

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.