Doubt on updating mappings

Elastic Stack Elasticsearch
1

Hi,

Had a few queries on using mappings. Mostly noob questions. I checked
the docs, but I was unable to find the answers.

I noticed this behaviour while testing ES.

  • On a particular index, I have 100 documents indexed.
  • After this, I update the mapping ( updated include_in_all & store
    for a few fields )
  • After this, I try a search query for _all fields and also with the
    'fields' argument
  • I noticed that my mapping update has not taken effect.
  • Now, I re-index the 100 documents and my queries work with the new mapping.

Am I doing the thing wrong or is this how it is designed ? As in, if
mapping is updated, then all documents have to be re-indexed for the
new mapping to be applicable on them.

On a similar topic, does this apply to index configuration also.
Say I have an index with 2 shards and 2 replicas each. I have X
documents indexed. If I re-configure the index with 3 shards and 4
replicas, will things be taken care of automatically ? Or should I
re-index the documents again ?

PS: Elastic Search is really a wonderful tool :slight_smile:

Regards,
Mahendra

http://twitter.com/mahendra

2

When updating mappings, you might have conflicts. Conflicts mean that you
are trying to set something that can't be set. By default in 0.6, conflicts
are ignored. I changed it to not be ignored in upcoming 0.7 so things will
be more obvious. You can always control that using the ignore_conflicts
parameter.

In your case, trying to change some settings on an existing mapper, it is
considered as a conflict. In the future, some settings might be applied
dynamically, though without reindexing, just for new data indexed.

Note, buy the way, that you can upgrade an existing single mapping into a
multi_field mapping, which is nice. Then, you can add more mapping on that
multi_field mapping (its not considered a conflict). But, this will only
apply to new data indexed.

cheers,
shay.banon

On Mon, May 3, 2010 at 10:53 AM, Mahendra M mahendra.m@gmail.com wrote:

Hi,

Had a few queries on using mappings. Mostly noob questions. I checked
the docs, but I was unable to find the answers.

I noticed this behaviour while testing ES.

  • On a particular index, I have 100 documents indexed.
  • After this, I update the mapping ( updated include_in_all & store
    for a few fields )
  • After this, I try a search query for _all fields and also with the
    'fields' argument
  • I noticed that my mapping update has not taken effect.
  • Now, I re-index the 100 documents and my queries work with the new
    mapping.

Am I doing the thing wrong or is this how it is designed ? As in, if
mapping is updated, then all documents have to be re-indexed for the
new mapping to be applicable on them.

On a similar topic, does this apply to index configuration also.
Say I have an index with 2 shards and 2 replicas each. I have X
documents indexed. If I re-configure the index with 3 shards and 4
replicas, will things be taken care of automatically ? Or should I
re-index the documents again ?

PS: Elastic Search is really a wonderful tool :slight_smile:

Regards,
Mahendra

http://twitter.com/mahendra

3

On Mon, May 3, 2010 at 1:58 PM, Shay Banon shay.banon@elasticsearch.com wrote:

In your case, trying to change some settings on an existing mapper, it is
considered as a conflict. In the future, some settings might be applied
dynamically, though without reindexing, just for new data indexed.

Sounds good. So, if I have to change the mapping, I will re-index the
data. Though in production, this will be a very very rare scenario.

Thanks,
Mahendra

4

Since elastic stores the raw data, it's possible the re-indexing could
be done entirely internally, no?

-N

On 3 May 2010, at 14:30, Mahendra M mahendra.m@gmail.com wrote:

On Mon, May 3, 2010 at 1:58 PM, Shay Banon <shay.banon@elasticsearch.com

wrote:
In your case, trying to change some settings on an existing mapper,
it is
considered as a conflict. In the future, some settings might be
applied
dynamically, though without reindexing, just for new data indexed.

Sounds good. So, if I have to change the mapping, I will re-index the
data. Though in production, this will be a very very rare scenario.

Thanks,
Mahendra

5

Yep, if storing the _source is enabled, then there can be an "index_to" API
that will completely reindex one index into another.

On Mon, May 3, 2010 at 7:10 PM, Nick Minutello nick.minutello@gmail.comwrote:

Since elastic stores the raw data, it's possible the re-indexing could be
done entirely internally, no?

-N

On 3 May 2010, at 14:30, Mahendra M mahendra.m@gmail.com wrote:

On Mon, May 3, 2010 at 1:58 PM, Shay Banon shay.banon@elasticsearch.com

wrote:

In your case, trying to change some settings on an existing mapper, it is
considered as a conflict. In the future, some settings might be applied
dynamically, though without reindexing, just for new data indexed.

Sounds good. So, if I have to change the mapping, I will re-index the
data. Though in production, this will be a very very rare scenario.

Thanks,
Mahendra