Issue With Index templates


(Shashank Jain) #1

Hi All,

I have a working pipeline with static ES Index as output .

But when i am trying to create index dynamically with index template , then indexes are not created .
Below are my configuration ,

output {
stdout { codec => rubydebug }
elasticsearch {
hosts => ["node1:9200","node2:9200", "node3:9200"]
index => "dynaminc_index-%{dd.MM.YYYY}"
manage_template => true
template_overwrite => true
template => "/etc/logstash/templates/dynamic.json"
template_name => "dynamic"
user => "logstash_internal"
password => "LS-INTERNAL"
}

   }

below are the content of template

{
"template": "dynamic",
"settings": {
"index": {
"number_of_shards": "5",
"number_of_replicas": "2",
"refresh_interval": "5s"

            }
    }

}

Please let me know what i am missing here .
It is bit critical issue so urgent suggestions would be appreciated .


(Christian Dahlqvist) #2

This template will only apply to an index named dynamic. To make it apply to your index, change this to an appropriate pattern, e.g. dynamic_index*.


(Shashank Jain) #3

Sorry , i didnt get you ,

where i have to make changes ,

In template file i.e. dynamic.json ??

or anywhere else ?


(Christian Dahlqvist) #4

You have to make the change in the index template. You can find an example of how to use patterns in the documentation.


(Shashank Jain) #5

dynamic.json
{
"template": "dynamic-",
"index_patterns": ["dynamic-
"],
"settings": {
"index": {
"number_of_shards": "5",
"number_of_replicas": "2",
"refresh_interval": "5s"

            }
    }

}

index => "dynamic-%{+dd.MM.YYYY}"
manage_template => true
#template_overwrite => true
template => "/etc/logstash/templates/dynamic.json"
template_name => "dynamic-*"
user => "logstash_internal"
password => "LS-INTERNAL"

changes done but same issue , have tried all combination .
am not getting what i am missing here .


(Christian Dahlqvist) #6

Looks like you are missing a wildcard (*) at the end.


(Shashank Jain) #7

nope it is there i have formatted the text so itz not apprearing


(Christian Dahlqvist) #8

What does the template stored in Elasticsearch look like?

You may need to enable this for the template to be replaced. The changed template will then apply only when new indices are created.


(Shashank Jain) #9

Same with enabling
template_overwrite => true ,

do i need to mention template management details in logstash.yml as well ,
manage_template => true
template_overwrite => true

{
"dynamic-": {
"order": 0,
"index_patterns": [
"dynamic-
"
],
"settings": {
"index": {
"number_of_shards": "5",
"number_of_replicas": "2",
"refresh_interval": "5s"
}
},
"mappings": {},
"aliases": {}
}
}


(Shashank Jain) #10

Can anyone help on this ??


(system) #11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.