# How to correctly parse and then enrich log data in Logstash?

**URL:** <https://discuss.elastic.co/t/how-to-correctly-parse-and-then-enrich-log-data-in-logstash/46244>\
**Category:** Logstash\
**Created:** [April 4, 2016, 1:53pm UTC](https://discuss.elastic.co/t/how-to-correctly-parse-and-then-enrich-log-data-in-logstash/46244 "2016-04-04T13:53:25Z")\
**Posts on this page:** 1\
**Showing post:** 9

<div class="post-metadata">

**Author:** ![nateuni](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nateuni/32/8512_2.png) [@nateuni](https://discuss.elastic.co/u/nateuni)\
**Post date:** [April 7, 2016, 10:16pm UTC](https://discuss.elastic.co/t/how-to-correctly-parse-and-then-enrich-log-data-in-logstash/46244/9 "2016-04-07T22:16:02Z")

</div>

Good point... as it wasn't initially apparent on how to do this.

Firstly I have changed the webserver to list dummy financial products (as this is what I am stating I am doing in my thesis). So if the URLs look different from my original post, that's why.

So I have a webserver that logs URLs that I want to parse in Logstash, so I can group them in on a per user basis in Kibana. I also want to get out of a lot of the crap out of the log file (see the last line)

An example log would be:

`<IP removed> - - [07/Apr/2016:12:41:38 +0000] "GET /product-category/banking/transaction-account/ HTTP/1.1" 200 6777 http://<IP removed>/product-category/banking/transaction-account/ wp-settings-1=libraryContent%3Dbrowse; wp-settings-time-1=1460008031; wordpress_test_cookie=WP+Cookie+check; wordpress_logged_in_cf8749745210721969771dfabf2df0ea=nathansturgess.pt%7C1460181231%7Czd5rFlzqVwPlbmfKEzJvzVhKeRoNLVfOJPbQNiHAZTs%7858084f4596e3a6863df44a88fbb59Cdc2d2145c62c3c6083519c1cfb8cf47ff2`

I want to match on the apache log data, the username in the cookie and on urls that could be in the following format:

```
/product-category/<category>/ an example would be /product-category/banking/
/product-category/<category>/<subcategory>/ an example would be /product-category/banking/transction-account/
/product/<product>/ example would be /product/everyday-account/

```

To do this matching I have the following filter snippet from my logstash.conf file:

```
filter {
    grok {
        match => { "message" => "\A%{COMMONAPACHELOG}(.*=(?<username>[^%]*)%)" }
    }
    grok {
        match => ["request", "/product-category\/(?<category>.*?)\/"]
    }
    grok {
        match => ["request", "/product-category\/.*?\/(?<subcategory>.*?)\/"]
    }
    grok {
        match => ["request", "/product\/(?<product>.*?)\/"]
    }
    mutate {
        remove_field => ["ident", "auth", "source", "hostname", "name", "host", "beat", "input_type"]
    }
}

```

[Here is a screenshot of a couple of lines of Kibana](http://postimg.org/image/fnxlvawx3/)

I then plan to enrich the logs with only the product with the category and subcategory and from lines with the subcategory to enrich the logs with the category. I need to figure out how to do this.

---

_[View the full topic](https://discuss.elastic.co/t/how-to-correctly-parse-and-then-enrich-log-data-in-logstash/46244)._