Mapping conflict!

Elastic Stack Elasticsearch
1

Dear community,

I have a mapping conflicts in across our indices (separated on a daily basis):

Mapping conflict! A field is defined as several types (string,
integer, etc) across the indices that match this pattern. You may still
be able to use these conflict fields in parts of Kibana, but they will
be unavailable for functions that require Kibana to know their type.
Correcting this issue will require re-indexing your data.

Yesterday I changed via logstash

% {NUMBER: bytes: int}

a field called "bytes" from the default type string to integer and today I have the issue.

How can I change the field type on the old indexes?

2

You need to reindex, you cannot change it dynamically like this.

3

I am new in the ELK world and I didn't understand how to do the reindex.

I found this on the guide:
https://www.elastic.co/guide/en/elasticsearch/guide/current/reindex.html
but it's not compresibile.

What are the steps to follow to reindex?

4

https://gist.github.com/markwalkom/8a7201e3f6ea4354ae06 is what I do, it uses Logstash.

5

And inspired by this great example, I wrote this blog post: http://david.pilato.fr/blog/2015/05/20/reindex-elasticsearch-with-logstash/

6

Thanks, great work! I have a few questions about some points of the blog post:

  • If I have multiple indexes (for example "logstash-YYYY.MM.DD"), can I do the same thing using as index "logstash- *" or do I have to reschedule the logstash input for each index?

  • If I don't have a new cluster, but I work on one as input and output, I will overwrite the old data or the data will be duplicated?

You talk about "first creating the mapping we want to apply before indexing any date", but considering the above (multiple indexes and clusters already existing), I can do the same this step?

7

I do individual cause I use this to merge daily indices into monthly ones.

If you're writing into the same cluster, you will want to put it into an entirely new index so you don't run the risk of keeping the old mapping.

1 Like