Mapping of index containing date in index(Ex. filebeat-2016.05.24)

Elastic Stack Elasticsearch
1

I am using the index pattern as "filebeat-[yyyy.mm.dd]". Some of the fields need to be analysed and hence I am uploading mapping.json template for the same with changes in it for required fields. The problem I am facing here is that, I am not able to do the mapping changes for the pattern "filebeat-*" and hence need to upload a new mapping file everyday. What can be done in such scenarios?

2

So you're saying that you have an index template that matches filebeat-*, yet a newly created filebeat-2016.05.24 index doesn't get the mappings from the template?

3

No,
Giving you an example. I have an index as filebeat-2016.05.24. It is having a field as "message" which needs to be analysed. To do the required changes I downloaded mapping.json file for the given index, made required changes and uploaded the template. Now when I receive new logs for the same index, my field "message" is analysed.

So, can we have a generalized template as "filebeat-*" instead of "filebeat-2016.05.24"? So that I don't need to upload the new mapping template for each date.

4

Giving you an example. I have an index as filebeat-2016.05.24. It is having a field as "message" which needs to be analysed. To do the required changes I downloaded mapping.json file for the given index, made required changes and uploaded the template.

But did you actually upload a template? It sounds like you just used the update mapping API. Which exact HTTP request did you make?

Now when I receive new logs for the same index, my field "message" is analysed.

You can't change the mappings of existing indexes. You have to reindex.

So, can we have a generalized template as "filebeat-*" instead of "filebeat-2016.05.24"? So that I don't need to upload the new mapping template for each date.

Yes, that's exactly the idea with index templates.

5

To get the field mapping I used following command:
curl -XGET 'http://localhost:9200/filebeat-2016.05.24/_mapping?pretty' > map.json

Now I made changes to map.json and uploaded it using following command:
curl http://localhost:9200/filebeat-2016.05.24 -X POST -d @map.json

I want to apply a generalized mapping which would be used by all the indices filebeat-*

6

Okay, then look into index templates. What you've been doing so far is unrelated to index templates.

7

Ok!! Thank you :slight_smile: