# How to extract index name from the uri field in logstash

**URL:** <https://discuss.elastic.co/t/how-to-extract-index-name-from-the-uri-field-in-logstash/135475>\
**Category:** Logstash\
**Created:** [June 12, 2018, 7:42am UTC](https://discuss.elastic.co/t/how-to-extract-index-name-from-the-uri-field-in-logstash/135475 "2018-06-12T07:42:14Z")\
**Posts on this page:** 18\
**Page:** 1

<div class="post-metadata">

**Author:** ![Santu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/santu/32/45072_2.png) [@Santu](https://discuss.elastic.co/u/Santu)\
**Post date:** [June 12, 2018, 7:42am UTC](https://discuss.elastic.co/t/how-to-extract-index-name-from-the-uri-field-in-logstash/135475/1 "2018-06-12T07:42:14Z")

</div>

Hi,

the api and index fields are not being populated in the request logs.  
for this i am using the custom pattern grok rules to extract the index and api values from the uri field like below. some how it doesn't works for me. any suggestions/help would be appreciated.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 12, 2018, 8:07am UTC](https://discuss.elastic.co/t/how-to-extract-index-name-from-the-uri-field-in-logstash/135475/2 "2018-06-12T08:07:29Z")

</div>

And what does an example input line look like?

When and how are the `esindex` and `esapi` fields set?

---

<div class="post-metadata">

**Author:** ![Santu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/santu/32/45072_2.png) [@Santu](https://discuss.elastic.co/u/Santu)\
**Post date:** [June 12, 2018, 8:31am UTC](https://discuss.elastic.co/t/how-to-extract-index-name-from-the-uri-field-in-logstash/135475/3 "2018-06-12T08:31:35Z")

</div>

sorry for confusing here. actual fields are index and api. i just renamed it to only index and api. please let me know if i have missed anything here,

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 12, 2018, 9:47am UTC](https://discuss.elastic.co/t/how-to-extract-index-name-from-the-uri-field-in-logstash/135475/4 "2018-06-12T09:47:12Z")

</div>

You missed answering both my questions.

---

<div class="post-metadata">

**Author:** ![Santu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/santu/32/45072_2.png) [@Santu](https://discuss.elastic.co/u/Santu)\
**Post date:** [June 12, 2018, 10:03am UTC](https://discuss.elastic.co/t/how-to-extract-index-name-from-the-uri-field-in-logstash/135475/5 "2018-06-12T10:03:02Z")

</div>

Hi Magnusbaeck,  
now i have updated the issue with proper exact field names. i hope now you understand my issue. and one more thing Grok pattern working fine in grok debugger but the same pattern is not working when running with logstash

please have a look and let me know what i am missing/need to add.

---

<div class="post-metadata">

**Author:** ![Santu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/santu/32/45072_2.png) [@Santu](https://discuss.elastic.co/u/Santu)\
**Post date:** [June 13, 2018, 6:53am UTC](https://discuss.elastic.co/t/how-to-extract-index-name-from-the-uri-field-in-logstash/135475/6 "2018-06-13T06:53:55Z")

</div>

my example input is like:: 1. /test/\_search

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 13, 2018, 7:21am UTC](https://discuss.elastic.co/t/how-to-extract-index-name-from-the-uri-field-in-logstash/135475/7 "2018-06-13T07:21:26Z")

</div>

What does a `stdout { codec => rubydebug }` output produce for an example input line? I haven't seen proof that the `uri` field contains the expected string.

---

<div class="post-metadata">

**Author:** ![Santu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/santu/32/45072_2.png) [@Santu](https://discuss.elastic.co/u/Santu)\
**Post date:** [June 13, 2018, 7:47am UTC](https://discuss.elastic.co/t/how-to-extract-index-name-from-the-uri-field-in-logstash/135475/8 "2018-06-13T07:47:58Z")

</div>

this is my output file looks like::

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 13, 2018, 8:44am UTC](https://discuss.elastic.co/t/how-to-extract-index-name-from-the-uri-field-in-logstash/135475/9 "2018-06-13T08:44:37Z")

</div>

Your grok filter appears to be the very first filter. Does the `uri` field really have the expected value at that time?

---

<div class="post-metadata">

**Author:** ![Santu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/santu/32/45072_2.png) [@Santu](https://discuss.elastic.co/u/Santu)\
**Post date:** [June 13, 2018, 8:49am UTC](https://discuss.elastic.co/t/how-to-extract-index-name-from-the-uri-field-in-logstash/135475/10 "2018-06-13T08:49:41Z")

</div>

Magnus i am not sure, also suspecting that uri parsing is not happening here. what should i do now? any suggetion?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 13, 2018, 9:07am UTC](https://discuss.elastic.co/t/how-to-extract-index-name-from-the-uri-field-in-logstash/135475/11 "2018-06-13T09:07:54Z")

</div>

I would expect you to know what kind of data enters Logstash. If it's JSON data that you're parsing with the json filter you obviously need to put the json filter before any filters that want to process fields found in the JSON string.

---

<div class="post-metadata">

**Author:** ![Santu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/santu/32/45072_2.png) [@Santu](https://discuss.elastic.co/u/Santu)\
**Post date:** [June 13, 2018, 11:02am UTC](https://discuss.elastic.co/t/how-to-extract-index-name-from-the-uri-field-in-logstash/135475/12 "2018-06-13T11:02:42Z")

</div>

yes Magnus. my data is in Json format and i do use below Json filters as well. but still it is not working.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 13, 2018, 11:05am UTC](https://discuss.elastic.co/t/how-to-extract-index-name-from-the-uri-field-in-logstash/135475/13 "2018-06-13T11:05:06Z")

</div>

Yes, but have you changed your configuration file so the json filter is listed **before** the grok filter?

---

<div class="post-metadata">

**Author:** ![Santu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/santu/32/45072_2.png) [@Santu](https://discuss.elastic.co/u/Santu)\
**Post date:** [June 13, 2018, 12:34pm UTC](https://discuss.elastic.co/t/how-to-extract-index-name-from-the-uri-field-in-logstash/135475/14 "2018-06-13T12:34:07Z")

</div>

Thanks for your reply Magnus.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 13, 2018, 1:37pm UTC](https://discuss.elastic.co/t/how-to-extract-index-name-from-the-uri-field-in-logstash/135475/15 "2018-06-13T13:37:13Z")

</div>

> grok {  
> json {  
> break\_on\_match =\> false  
> keep\_empty\_captures =\> true  
> match =\> ["uri", "(^/)"]  
> match =\> ["uri", "(/)"]  
> tag\_on\_failure =\>   
> }  
> }

No! Nowhere did I suggest you put the grok filter **inside** the json filter.

Look, this is very simple. Filters are executed in the order listed in the configuration. If you have one filter that creates a new field like `uri` (e.g. by parsing a JSON string) and another filter that attempts to parse that field the second filter needs to come after. Since you still haven't provided an example line from your log you've left me guessing at how your configuration is supposed to work. Over and out, and good luck.

---

<div class="post-metadata">

**Author:** ![Santu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/santu/32/45072_2.png) [@Santu](https://discuss.elastic.co/u/Santu)\
**Post date:** [June 13, 2018, 3:26pm UTC](https://discuss.elastic.co/t/how-to-extract-index-name-from-the-uri-field-in-logstash/135475/16 "2018-06-13T15:26:03Z")

</div>

Magnus sorry for missing the input log file here. i thought already i have attached. please find the same here.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 13, 2018, 6:00pm UTC](https://discuss.elastic.co/t/how-to-extract-index-name-from-the-uri-field-in-logstash/135475/17 "2018-06-13T18:00:14Z")

</div>

Before you worry about the filters, you should fix your input so that it correctly assembles events. The log you show has the timestamp on the line after the JSON. So your multiline codec configuration should have negate =\> false.

If you use negate =\> true then the first line of the log (the JSON) will appear in a separate event, which will massively confuse you as you try to debug the configuration.

Once you are assembling events correctly I would recommend you start with a dissect filter to break out the JSON from the timestamp, log level, etc. Then a json filter to parse the JSON.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 11, 2018, 6:00pm UTC](https://discuss.elastic.co/t/how-to-extract-index-name-from-the-uri-field-in-logstash/135475/18 "2018-07-11T18:00:14Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
