# Grok URI extract **URL:** \ **Category:** Logstash\ **Created:** [July 6, 2016, 2:16pm UTC](https://discuss.elastic.co/t/grok-uri-extract/54836 "2016-07-06T14:16:15Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![carrot](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/carrot/32/44335_2.png) [@carrot](https://discuss.elastic.co/u/carrot)\ **Post date:** [July 6, 2016, 2:16pm UTC](https://discuss.elastic.co/t/grok-uri-extract/54836/1 "2016-07-06T14:16:15Z")
I've been searching and reading documentation but I can't seem to find how to extract parameters from a URI. The best I could do so far is : ``` grok { match => ["url", "%{URIPROTO:uri_proto}://(?:%{USER:user}(?::[^@]*)?@)?(?:%{URIHOST:uri_domain})?(?:%{URIPATHPARAM:uri_param})?" ] } ``` Result ``` "url" => "http://hostname.domain.tld/_astats?application=&inf.name=eth0" "uri_proto" => "http", "uri_domain" => "hostname.domain.tld", "uri_param" => "/_astats?application=&inf.name=eth0" ``` I'd like to separate all the components in the following field... "uri\_domain, uri\_path, uri\_root", I would also like to extract the query string into sub fields like "query\_params: application, [inf.name](http://inf.name)". For example, how do I expand URIPATH and URIPARAM vs URIPATHPARAM? Why isn't all of them expanded when I call just "URI" through these GROK definitions? # paths ``` PATH (?:%{UNIXPATH}|%{WINPATH}) UNIXPATH (/([\w_%!$@:.,~-]+|\\.)*)+ TTY (?:/dev/(pts|tty([pq])?)(\w+)?/?(?:[0-9]+)) WINPATH (?>[A-Za-z]+:|\\)(?:\\[^\\?*]*)+ URIPROTO [A-Za-z]+(\+[A-Za-z+]+)? URIHOST %{IPORHOST}(?::%{POSINT:port})? # uripath comes loosely from RFC1738, but mostly from what Firefox # doesn't turn into %XX URIPATH (?:/[A-Za-z0-9$.+!*'(){},~:;=@#%_\-]*)+ #URIPARAM \?(?:[A-Za-z0-9]+(?:=(?:[^&]*))?(?:&(?:[A-Za-z0-9]+(?:=(?:[^&]*))?)?)*)? URIPARAM \?[A-Za-z0-9$.+!*'|(){},~@#%&/=:;_?\-\[\]<>]* URIPATHPARAM %{URIPATH}(?:%{URIPARAM})? URI %{URIPROTO}://(?:%{USER}(?::[^@]*)?@)?(?:%{URIHOST})?(?:%{URIPATHPARAM})? ``` Thanks for any information. ---
**Author:** ![bhatch](https://avatars.discourse-cdn.com/v4/letter/b/e95f7d/32.png) [@bhatch](https://discuss.elastic.co/u/bhatch)\ **Post date:** [July 6, 2016, 6:08pm UTC](https://discuss.elastic.co/t/grok-uri-extract/54836/2 "2016-07-06T18:08:02Z")
If you run this GROK statement on your uri\_param field you can get these results. This will break it up into two separate components. No idea if it is the best method, but it has worked ok for us. GROK `%{GREEDYDATA:uri_stem}\?%{GREEDYDATA:uri_query}` Results ![](https://us1.discourse-cdn.com/elastic/original/2X/7/7eacd1ecb046cb48c75fe5fa55dbc4155a05fbcf.png) If you want the query portion broken down into separate components you can try using the KV filter. ``` kv { source => "uri_query" field_split => "&" prefix => "query_" } ``` So now we can take the uri\_query we generated above (with added parameters). `{"uri_query":"application=&inf.name=eth0&test1=blah&test2=blahblahblah"}` And when we run it through that KV filter we get: ``` { "uri_query" => "application=&inf.name=eth0&test1=blah&test2=blahblahblah", "query_inf.name" => "eth0", "query_test1" => "blah", "query_test2" => "blahblahblah" } ``` You can then drop the original uri\_query field or keep it around. If you want to limit which fields get created use the include\_keys so that you don't accidentally create hundreds of new fields in your cluster. ---
**Author:** ![carrot](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/carrot/32/44335_2.png) [@carrot](https://discuss.elastic.co/u/carrot)\ **Post date:** [July 7, 2016, 2:20pm UTC](https://discuss.elastic.co/t/grok-uri-extract/54836/3 "2016-07-07T14:20:53Z")
Thanks, this was really helpful... I found the "target" parameter to put the query string into a container. Exactly what I needed. I couldn't put all my "match" statement in the same grok, had to be separated... I'm still baffled that I can't just call "URI" to extract everything based on the logstash-grok pattern here : [https://github.com/hpcugent/logstash-patterns/blob/master/files/grok-patterns](https://github.com/hpcugent/logstash-patterns/blob/master/files/grok-patterns) ``` filter { grok { match => ["url", "%{URIPROTO:uri_proto}://(?:%{USER:user}(?::[^@]*)?@)?(?:%{URIHOST:uri_domain})?(?:%{URIPATHPARAM:uri_param})?" ] } grok { match => ["uri_param", "%{GREEDYDATA:uri_path}\?%{GREEDYDATA:uri_query}"] } kv { source => "uri_query" field_split => "&" target => "query" } } ``` The result was ``` "url" => "http://cdn1cdedge0001.coxlab.net/_astats?application=&inf.name=eth0", "uri_proto" => "http", "uri_domain" => "cdn1cdedge0001.coxlab.net", "uri_param" => "/_astats?application=&inf.name=eth0", "uri_path" => "/_astats", "uri_query" => "application=&inf.name=eth0", "query" => { "inf.name" => "eth0" } ``` Then I can simply remove "uri\_query" since it's now duplicate. Although I haven't tested this into ES yet. ---
**Author:** ![bhatch](https://avatars.discourse-cdn.com/v4/letter/b/e95f7d/32.png) [@bhatch](https://discuss.elastic.co/u/bhatch)\ **Post date:** [July 7, 2016, 3:02pm UTC](https://discuss.elastic.co/t/grok-uri-extract/54836/4 "2016-07-07T15:02:41Z")
I think for whatever reason the URI Grok statement just wasn't designed how you or I want it to work. A comment on the second GROK statement. As long as there are query parameters (Anything after the question mark) this will work fine. However if you have a URI that does not have this this GROK statement will not match. That means that uri\_query and uri\_path will not be populated. uri\_query is a given since there is no query, but depending on your usage you may still want uri\_path to show up. At this point it would hold the same value as uri\_param, but for aggregations of searching that may not matter. If you still want uri\_path to be populated regardless you can try using this version. If the first match doesn't work, it will go onto the second statement. You could also try using conditionals to check if the field doesn't exist and add it, but this method seems pretty simple. ``` grok { break_on_match => true match => [ "uri_param", "%{GREEDYDATA:uri_path}\?%{GREEDYDATA:uri_query}" , "uri_param","%{GREEDYDATA:uri_path}" ] } ``` ---
**Author:** ![munotshubham](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/munotshubham/32/22663_2.png) [@munotshubham](https://discuss.elastic.co/u/munotshubham)\ **Post date:** [May 24, 2017, 5:36pm UTC](https://discuss.elastic.co/t/grok-uri-extract/54836/5 "2017-05-24T17:36:39Z")
Hey, how do I split uri\_stem? ---
**Author:** ![bhatch](https://avatars.discourse-cdn.com/v4/letter/b/e95f7d/32.png) [@bhatch](https://discuss.elastic.co/u/bhatch)\ **Post date:** [May 31, 2017, 3:31pm UTC](https://discuss.elastic.co/t/grok-uri-extract/54836/6 "2017-05-31T15:31:13Z")
Which uri\_stem? We mentioned a few here. And how do you want it split up? ---
**Author:** ![munotshubham](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/munotshubham/32/22663_2.png) [@munotshubham](https://discuss.elastic.co/u/munotshubham)\ **Post date:** [May 31, 2017, 3:54pm UTC](https://discuss.elastic.co/t/grok-uri-extract/54836/8 "2017-05-31T15:54:32Z")
Hey thanks for replying. I was able to figure that out later. Here is the link of my post in which I had an issue. If anyone still looking similar problem and the solution. > [@KV filter not working](https://discuss.elastic.co/t/kv-filter-not-working/87019): > > config file: filter { if [type] == "syslog" { grok { match =\> { "message" =\> "%{SYSLOGTIMESTAMP:syslog\_timestamp} %{SYSLOGHOST:syslog\_hostname} %{DATA:syslog\_program}(?:\[%{POSINT:syslog\_pid}\])?: %{GREEDYDATA:syslog\_message}" } add\_field =\> ["received\_at", "%{@timestamp}"] add\_field =\> ["received\_from", "%{host}"] } syslog\_pri { } date { match =\> ["syslog\_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"] } } if [message] =~ "^#" { dro… ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:26am UTC](https://discuss.elastic.co/t/grok-uri-extract/54836/9 "2017-07-06T04:26:09Z")