# Custom index not creating at ElasticSearch cluster

**URL:** https://discuss.elastic.co/t/custom-index-not-creating-at-elasticsearch-cluster/61016
**Category:** Logstash
**Created:** [September 20, 2016, 1:18pm UTC](https://discuss.elastic.co/t/custom-index-not-creating-at-elasticsearch-cluster/61016 "2016-09-20T13:18:54Z")
**Posts on this page:** 10
**Page:** 1

<div class="post-metadata">

### Author: ![Phani\_Nadiminti](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/phani_nadiminti/32/45381_2.png) [@Phani\_Nadiminti](https://discuss.elastic.co/u/Phani_Nadiminti)
#### Post date: [September 20, 2016, 1:18pm UTC](https://discuss.elastic.co/t/custom-index-not-creating-at-elasticsearch-cluster/61016/1 "2016-09-20T13:18:54Z")

</div>

HI All,

I am using ES 1.5.2 with logstash 2.4.0 . i have the following configurations to pull data from kafka topic and index to ES.

input{  
kafka {  
zk\_connect =\> ":2181"  
group\_id =\> "logstash"  
topic\_id =\> "test"  
consumer\_threads =\> 2  
}  
}  
output {  
elasticsearch {  
hosts =\> ["10.10.1.2:9200"]  
index =\> "test123"  
}  
stdout { codec =\> rubydebug }  
}

I given test123 as my index name but custom index not creating at ES rather it is creating default "test" index and pushing data to that index only.please suggest me where i am doing wrong in above configurations.

Thanks  
phani

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [September 20, 2016, 2:02pm UTC](https://discuss.elastic.co/t/custom-index-not-creating-at-elasticsearch-cluster/61016/2 "2016-09-20T14:02:31Z")

</div>

Not sure what you mean. By default Logstash doesn't write to an index named "test", so if Logstash is sending events to an index with that name it's because you have configured it that way.

---

<div class="post-metadata">

### Author: ![Phani\_Nadiminti](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/phani_nadiminti/32/45381_2.png) [@Phani\_Nadiminti](https://discuss.elastic.co/u/Phani_Nadiminti)
#### Post date: [September 20, 2016, 2:06pm UTC](https://discuss.elastic.co/t/custom-index-not-creating-at-elasticsearch-cluster/61016/3 "2016-09-20T14:06:13Z")

</div>

Thank you @magnusbaeck for quick reply i am trying to write documents to custom index named as "test123" but it is not creating at elasticsearch side.

phani

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [September 20, 2016, 2:08pm UTC](https://discuss.elastic.co/t/custom-index-not-creating-at-elasticsearch-cluster/61016/4 "2016-09-20T14:08:38Z")

</div>

Are you reading events from Kafka then, i.e. is it an input or an output problem?

---

<div class="post-metadata">

### Author: ![Phani\_Nadiminti](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/phani_nadiminti/32/45381_2.png) [@Phani\_Nadiminti](https://discuss.elastic.co/u/Phani_Nadiminti)
#### Post date: [September 20, 2016, 2:10pm UTC](https://discuss.elastic.co/t/custom-index-not-creating-at-elasticsearch-cluster/61016/5 "2016-09-20T14:10:05Z")

</div>

able to consume data from kafka but writing to es i am getting problem it is problem with output it seems.

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [September 20, 2016, 2:17pm UTC](https://discuss.elastic.co/t/custom-index-not-creating-at-elasticsearch-cluster/61016/6 "2016-09-20T14:17:19Z")

</div>

Have a look in the Logstash logs.

---

<div class="post-metadata">

### Author: ![Phani\_Nadiminti](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/phani_nadiminti/32/45381_2.png) [@Phani\_Nadiminti](https://discuss.elastic.co/u/Phani_Nadiminti)
#### Post date: [September 20, 2016, 2:50pm UTC](https://discuss.elastic.co/t/custom-index-not-creating-at-elasticsearch-cluster/61016/7 "2016-09-20T14:50:36Z")

</div>

@magnusbaeck i have looked at the logs but it seems logstash not generating logs when i start logstash with kafka configuration it is showing like this.

./logstash -f kafka-topic.conf  
Settings: Default pipeline workers: 2  
Pipeline main started

no logs hitting to logstash server.i have used .tar for logstash rather than service

please advise me.

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [September 20, 2016, 4:35pm UTC](https://discuss.elastic.co/t/custom-index-not-creating-at-elasticsearch-cluster/61016/8 "2016-09-20T16:35:39Z")

</div>

Increase the log level by starting with `--verbose` or even `--debug`.

---

<div class="post-metadata">

### Author: ![Phani\_Nadiminti](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/phani_nadiminti/32/45381_2.png) [@Phani\_Nadiminti](https://discuss.elastic.co/u/Phani_Nadiminti)
#### Post date: [September 21, 2016, 12:11pm UTC](https://discuss.elastic.co/t/custom-index-not-creating-at-elasticsearch-cluster/61016/9 "2016-09-21T12:11:29Z")

</div>

Hi @magnusbaeck,

Thank you for suggesting and i have checked using --verbose there i found like mapping i created loading or not . it seems the default logstash mapping is applying with document\_type logs instead of my custom one.The following is the configuration i applied to get it work.

action =\> "index"  
hosts =\> ["10.10.1.36:9200"]  
index =\> "logs\_test"  
document\_type =\> "lv2"  
template =\> "/home/test\_bkp.json"  
template\_name =\> "kafka\_template"  
template\_overwrite =\> 'true'

Explicit document type , template\_overwrite =\> 'true' did the trick for me and i am loading custom index template from template.

Now indexing data from kafka topic to es is working and how can we create monthly index using the same configurations please suggest me.

Thanks  
phani

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 4:37am UTC](https://discuss.elastic.co/t/custom-index-not-creating-at-elasticsearch-cluster/61016/10 "2017-07-06T04:37:37Z")

</div>


