# Index not created by Logstash

**URL:** <https://discuss.elastic.co/t/index-not-created-by-logstash/141231>\
**Category:** Logstash\
**Created:** [July 23, 2018, 5:40pm UTC](https://discuss.elastic.co/t/index-not-created-by-logstash/141231 "2018-07-23T17:40:15Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Albert](https://avatars.discourse-cdn.com/v4/letter/a/48db29/32.png) [@Albert](https://discuss.elastic.co/u/Albert)\
**Post date:** [July 23, 2018, 5:40pm UTC](https://discuss.elastic.co/t/index-not-created-by-logstash/141231/1 "2018-07-23T17:40:16Z")

</div>

Hi all,

I'm following this blog ([https://www.elastic.co/blog/monitoring-the-search-queries](https://www.elastic.co/blog/monitoring-the-search-queries)) to capture search queries of Elastic Search that is installed on a MediaWiki site with a CirrusSearch extension using the following versions:

- logstash 5.6.10
- packetbeat version 5.6.10 (amd64), libbeat 5.6.10
- elasticsearch Version: 5.6.10, Build: b727a60/2018-06-06T15:48:34.860Z, JVM: 1.8.0\_141
- kibana 5.6.10

All is running on the same server (linux redhat).

As logstash conf file, I'm using the same as indicated by the blog:  
\</\>  
input {  
beats {  
port =\> 5044  
}  
}  
filter {  
if "search" in [request]{  
grok {  
match =\> { "request" =\> "._\n{(?\<query\_body\>._)"}  
}  
grok {  
match =\> { "path" =\> "/(?.\*)/\_search"}  
}  
if [index] {  
} else {  
mutate {  
add\_field =\> { "index" =\> "All" }  
}  
}  
mutate {  
update =\> { "query\_body" =\> "{%{query\_body}" }  
}  
}  
}  
output {  
if "search" in [request] and "ignore\_unmapped" not in [query\_body]{  
elasticsearch {  
hosts =\> ["[http://localhost:9200](http://localhost:9200)"]  
#index =\> "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"  
}  
}  
}  
\</\>

To the elasticsearch.yml file the following line is added:  
action.auto\_create\_index: logstash-\*

However, no logstash-\* indices are generated when searching on the wiki.  
E.g. curl localhost:9200/\_cat/indices?v  
\</\>  
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size  
green open csdms\_wiki-mw\_\_general\_first S8fUiAoKRpKjh4lA163hJQ 4 0 5202 1 19.5mb 19.5mb  
green open csdms\_wiki-mw\_\_content\_first 81VvhnP2SE2hzYFtgqn0DQ 4 0 11339 1908 125.5mb 125.5mb  
green open mw\_cirrus\_metastore\_first EGfxsk15Qp61eLw4ydIr\_A 1 0 3 2 6.2kb 6.2kb  
yellow open logstash-x mdA01kCNTUilnRYi2PisAg 5 1 1 0 4.6kb 4.6kb  
yellow open .kibana D9w9nJ6ORf6y4kwAzdGSwg 1 1 3 0 22.3kb 22.3kb  
\</\>

(The "logstash-x" was created by hand as a testcase). Any idea what I'm doing wrong, why no logstash-\* are generated?

Thank you!,  
Albert

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [July 25, 2018, 8:14pm UTC](https://discuss.elastic.co/t/index-not-created-by-logstash/141231/2 "2018-07-25T20:14:52Z")

</div>

Have you looked in the logs for Logstash and Packetbeat for clues?

---

<div class="post-metadata">

**Author:** ![Albert](https://avatars.discourse-cdn.com/v4/letter/a/48db29/32.png) [@Albert](https://discuss.elastic.co/u/Albert)\
**Post date:** [July 25, 2018, 10:30pm UTC](https://discuss.elastic.co/t/index-not-created-by-logstash/141231/3 "2018-07-25T22:30:38Z")

</div>

Thank you Magnus,

Yes, logs don't provide any error. What I did figure out is that the provided logstash conf file given in the blog doesn't work in my case. Specifically the "if" statement in the output part:

\</\>  
if "search" in [request] and "ignore\_unmapped" not in [query\_body]{  
\</\>

but also the filter needs adjustments.

The if statement in the output part makes that no logstash-\* index is generated (so no output is generated). Probably because the [request] field is not sufficient. Maybe this should be replaced for me by [http][request][params]. I'm not familiar with logstash conf files so this will be trial and error. But it looks like my problem is solved. Indices are created by Logstash when removing filter and if statement (now generating tons of data, so have to turn it off).

Thanks,  
Albert.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 22, 2018, 10:44pm UTC](https://discuss.elastic.co/t/index-not-created-by-logstash/141231/4 "2018-08-22T22:44:11Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.