How to Parse url.query into Different Fields?

Elastic Stack Beats
1

ELK 7.3 and Filebeat 7.3

I'm using the IIS module currently, and everything is parsing great. I would like to further parse the url.query section of this line, indicated in bold:

2019-09-24 17:14:04 10.202.225.10 GET /Sso/Internal/SignOn.aspx fi=10_2084e051-0184-4746-a017-558fdf9a99a1&customer=3 443 - 10.202.225.254

I would like to separate it into 3 sections:

  1. fi=10 the underscore following this would be the delimiter. Title would be "Client"
  2. the middle section of stuff, the ampsersand being the delimiter. Title being "Widget"
  3. customer=3 would be the last section. Title being "Customer"

Since I'm using the built in IIS module, I don't know where to put this additional parsing language, or frankly how to make it happen at all.

2

Hello, thanks for the question about filebeat. In your configuration, is filebeat shipping directly to elasticsearch, or does filebeat first ship data to logstash for enrichment or filtering. If you're using logstash, this could be a possible good portion of the pipeline to split about the url.query string.

If you're using logstash, I would recommend looking at the grok filter plugin:
https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html

3

I am shipping directly to Elasticsearch.

4

Hi, thanks for the response. I am shipping directly to Elasticsearch, is shipping to logstash the best way to split url.query or can it be achieved using the ingest pipeline in filebeat?

Any advice is appreciated, I'm still a newbie.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.