# Defining index mappings on ElasticSearch **URL:** \ **Category:** Elasticsearch\ **Tags:** docker, language-clients\ **Created:** [September 16, 2020, 6:50am UTC](https://discuss.elastic.co/t/defining-index-mappings-on-elasticsearch/248774 "2020-09-16T06:50:41Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![szanin](https://avatars.discourse-cdn.com/v4/letter/s/a9a28c/32.png) [@szanin](https://discuss.elastic.co/u/szanin)\ **Post date:** [September 16, 2020, 6:50am UTC](https://discuss.elastic.co/t/defining-index-mappings-on-elasticsearch/248774/1 "2020-09-16T06:50:41Z")
Hi there, This is my first post here, so I apologise in advance if I'm not too clear or if my question has been asked already. I have ElasticSearch running in a Docker cluster, I use it to search through a [ASP.NET](http://ASP.NET) core API that queries ES. I would like to define the index mappings for ES at cluster creation, running a job in my docker-compose that uses a .NET console app to put the requests on ES to define the mappings. I have seen that using templates should do the job for me, but I can't find any examples for best practices in NEST or ElasticSearch-NET so I thought I'd ask here. My idea would be to simply define a template that is applied to any new index that is created. Then I run an external job that seeds ES and any new index will have the same mapping applied to it whenever a new document needs to go in a new index. I also want to define an alias that is common to all the new indexes. The mappings I want to define are as below: ```auto { "settings": { "analysis": { "filter": { "autocomplete_filter": { "type": "edge_ngram", "min_gram": 1, "max_gram": 20 } }, "analyzer": { "autocomplete": { "type": "custom", "tokenizer": "standard", "filter": [ "lowercase", "autocomplete_filter" ] } } } }, "mappings": { "dynamic": false, "properties": { "Name": { "type": "keyword" }, "Cusip": { "type": "keyword" }, "ISIN": { "type": "keyword" }, "Ticker": { "type": "keyword" }, "suggest": { "type": "completion", "analyzer": "autocomplete", "search_analyzer": "standard" } } } } ``` And for the alias I need to use something like this: ```auto { "actions" : [ { "add" : { "index" : "{index}", "alias" : "product" } } ] } ``` How do I package it together into a template? Is using a template the right way to do this? How do I ensure that all new indexes have these settings? Thanks in advance for your help Simon ---
**Author:** ![mykael](https://avatars.discourse-cdn.com/v4/letter/m/ec9cab/32.png) [@mykael](https://discuss.elastic.co/u/mykael)\ **Post date:** [September 17, 2020, 1:16am UTC](https://discuss.elastic.co/t/defining-index-mappings-on-elasticsearch/248774/2 "2020-09-17T01:16:10Z")
At the top of my templates I have: ``` { "index_patterns" : ["test_data_index_hlq-*"], "template" : { "settings" : { "number_of_shards" : 1 }, "mappings" : { "properties" : { "@timestamp" : { "type" : "date" }, ``` That seems to be enough to tell it to use the template for any index that matches the pattern. The trailing - on the pattern may be significant - or maybe you just have to be consistent with kibana and transforms. ---
**Author:** ![szanin](https://avatars.discourse-cdn.com/v4/letter/s/a9a28c/32.png) [@szanin](https://discuss.elastic.co/u/szanin)\ **Post date:** [September 21, 2020, 12:55am UTC](https://discuss.elastic.co/t/defining-index-mappings-on-elasticsearch/248774/3 "2020-09-21T00:55:49Z")
In the end is solved this using the LowLevel Elasticsearch-net client. I have a config file that contains the JSON request for the template: ```auto {"index_patterns": [ "*" ], "settings": { "analysis": { "filter": { "autocomplete_filter": { "type": "edge_ngram", "min_gram": 1, "max_gram": 20 } }, "analyzer": { "autocomplete": { "type": "custom", "tokenizer": "standard", "filter": [ "lowercase", "autocomplete_filter" ] } } } }, "mappings": { "dynamic": false, "properties": { "Name": { "type": "keyword" }, "field1": { "type": "keyword" }, "field2": { "type": "keyword" }, "field3": { "type": "keyword" }, "suggest": { "type": "completion", "analyzer": "autocomplete", "search_analyzer": "standard" } } }, "aliases" : { "product" : {} } } ``` And I send the request for the template mapping like this: ```auto // Send a PUT request containing the template var postMapping = _client.LowLevel.DoRequest(HttpMethod.PUT, "_template/product_template", PostData.String(template.ToString())); ``` ---
**Author:** ![szanin](https://avatars.discourse-cdn.com/v4/letter/s/a9a28c/32.png) [@szanin](https://discuss.elastic.co/u/szanin)\ **Post date:** [September 21, 2020, 1:00am UTC](https://discuss.elastic.co/t/defining-index-mappings-on-elasticsearch/248774/4 "2020-09-21T01:00:57Z")
I still have to solve how to ensure that only new indices get my template, and not the metrics or logs indices. If I define the "index\_pattern" to be "\*" that will apply to all new indices. Is there a way to specify that I don't want to apply the template to indices starting with "."? The examples for index\_pattern are not very helpful, somehow we expect indices that share the same template to start with the same letter, like "t\*" etc. In my use case I have a list of more than 30 types of products that I want to share the same settings, and I don't want to add every product type to the index\_pattern list. On top of that, I want to be able to use the same template for all the new products that might be added in the future, since I know that my ES cluster will always contain objects that belong to that category. Is there a way to achieve this, without changing the index name of all products? So far the only solution I've found is to prepend a string to the index names so that I have them be like "q-abc", "q-xyz" etc. The index pattern matching seems to only allow for wildcard matching, is there a plan to implement something like RegEx or more advanced pattern matching solutions? @Russ_Cam ---
**Author:** ![forloop](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/forloop/32/9021_2.png) [@forloop](https://discuss.elastic.co/u/forloop)\ **Post date:** [September 21, 2020, 2:07am UTC](https://discuss.elastic.co/t/defining-index-mappings-on-elasticsearch/248774/5 "2020-09-21T02:07:13Z")
> [@szanin](#): > > // Send a PUT request containing the template > > ```auto > var postMapping = > _client.LowLevel.DoRequest(HttpMethod.PUT, "_template/product_template", PostData.String(template.ToString())); > > ``` You can use the low level client API method, and return a high level response if using the low level client exposed on the high level client ```auto var client = new ElasticClient(); var json = @"{""index_patterns"": [ ""*"" ], ""settings"": { ""analysis"": { ""filter"": { ""autocomplete_filter"": { ""type"": ""edge_ngram"", ""min_gram"": 1, ""max_gram"": 20 } }, ""analyzer"": { ""autocomplete"": { ""type"": ""custom"", ""tokenizer"": ""standard"", ""filter"": [ ""lowercase"", ""autocomplete_filter"" ] } } } }, ""mappings"": { ""dynamic"": false, ""properties"": { ""Name"": { ""type"": ""keyword"" }, ""field1"": { ""type"": ""keyword"" }, ""field2"": { ""type"": ""keyword"" }, ""field3"": { ""type"": ""keyword"" }, ""suggest"": { ""type"": ""completion"", ""analyzer"": ""autocomplete"", ""search_analyzer"": ""standard"" } } }, ""aliases"" : { ""product"" : { } } } "; var response = client.LowLevel.Indices.PutTemplateForAll("product_template", json); ``` > [@szanin](#): > > I still have to solve how to ensure that only new indices get my template, and not the metrics or logs indices. If I define the "index\_pattern" to be "\*" that will apply to all new indices. Is there a way to specify that I don't want to apply the template to indices starting with "."? > > The examples for index\_pattern are not very helpful, somehow we expect indices that share the same template to start with the same letter, like "t\*" etc. In my use case I have a list of more than 30 types of products that I want to share the same settings, and I don't want to add every product type to the index\_pattern list. On top of that, I want to be able to use the same template for all the new products that might be added in the future, since I know that my ES cluster will always contain objects that belong to that category. Is there a way to achieve this, without changing the index name of all products? I think index patterns are intended to work on the basis that you know and control the names of indices created, so that index templates apply only to a subset of indices. Configuring an index template with the `*` pattern indicates that you want to apply a template to all indices, but that doesn't sound like what you want to do. The `order` parameter can be used to define the order in which multiple templates apply, which _may_ help, ---
**Author:** ![szanin](https://avatars.discourse-cdn.com/v4/letter/s/a9a28c/32.png) [@szanin](https://discuss.elastic.co/u/szanin)\ **Post date:** [September 21, 2020, 3:32am UTC](https://discuss.elastic.co/t/defining-index-mappings-on-elasticsearch/248774/6 "2020-09-21T03:32:18Z")
Thanks so much for your reply! Your suggestion to use: `client.LowLevel.Indices.PutTemplateForAll("product_template", json);` seems a lot better, to be honest I couldn't find any reference to this method in the .NET client after version 1.x, maybe the docs could be updated to make it more clear? Regarding the pattern matching issue, I don't think the order would solve it because it would still be applied to all indices, and that causes a fatal error when spinning up Kibana because the index mappings for the metrics/logs indices are misconfigured. I'll follow @warkolm's advice [here](https://discuss.elastic.co/t/template-index-pattern-matching/249289/2) and open a feature request for a "not match" index pattern matching. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [October 19, 2020, 3:32am UTC](https://discuss.elastic.co/t/defining-index-mappings-on-elasticsearch/248774/7 "2020-10-19T03:32:20Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.