How to use the SAME mapping for all similar indices, and then for all types in the index?

Elastic Stack Elasticsearch
1

ES 2.4.1

I create a daily logstash index to meet my needs, which has the form "logstash-application-customer-YYYY.MM.dd" where the YYYY.MM.dd is of course the value that meets the "daily" requirement. I can have many applications and many customers, but those do not change daily.

So per the API I got my mapping and got this sample snippet

{
  "logstash-applicationX-customer-2017.03.29" : {
    "mappings" : {
      "appX_type_log_1" : {
          # Mappings for log_1 types
       }
      "appX_type_log_2" : {
          # Mappings for log_2 types
       }
    }
  }
}

I need different types because my Grok filters are conditioned on these types, though I do have the same fields for ALL the types for each application.

This means that I'll have a different mapping for an index created on a different day, which will look exactly the same as the above mapping, correct? Also I noticed that the type_log_1 mappings, are EXACTLY the same as the type_log_2 mappings. So how can I create a mapping that essentially does this for applicationX

{
  "logstash-applicationX-*" : {
    "mappings" : {
      "appX_type_*" : {
          # Mappings for ALL types for logstash-applicationX-*
       }
    }
  }
}

I can then have a different mapping for applicationY, which has different types.

{
  "logstash-applicationY-*" : {
    "mappings" : {
      "appY_type_*" : {
          # Mappings for ALL types for logstash-applicationY-*
       }
    }
  }
}

Does that make sense?

2

Hi,

What about template?

https://www.elastic.co/guide/en/elasticsearch/reference/5.3/indices-templates.html

3

@gabriel_tessier, thanks. I started reading about templates per your post. I have some questions. If I have this template...

{
  "template" : "logstash-appX-*",
  "mappings" : {
    "appX-type-*" : {
       # some mappings
    }
  }
}

What is the significance of the template name, logstash-appX-* in this case. Does this mean that ALL indices that have the form ogstash-appX-, e.g., logstash-appX-2017.03.29, logstash-appX-2017.03.30, will use this template? However, indices with logstash-appY- will NOT, and will need/want another template for those indices? IOW, in general, how do I associate an indices to a certain template?

Will the appX-type-* mapping, with the * in the name, apply to all types of that form, that is appX-type-0, appX-type-1, etc?

Thanks.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.