Changing default mapping

Elastic Stack Elasticsearch
1

I have the following default mapping:

{"company" : {"item" : {"properties" : {
"model" : {"type" : "string"},
"itemID" : {"type" : "string"},
"listingDate" : {"format" : "dateOptionalTime", "type" :
"date"},
"condition" : {"type" : "string"},
"type" : {"type" : "string"},
"mainImage" : {"type" : "string"},
"percOff" : {"type" : "string"},
"size" : {"type" : "string"},
"username" : {"type" : "string"},
"category" : {"type" : "string"},
"title" : {"type" : "string"},
"price" : {"type" : "string"},
"msrp" : { "type" : "string"},
"description" : { "type" : "string"},
"brand" : { "type" : "string"}}}}}

Then I go and change some values and I'll need to sort them and run
queries with ranges on some of them with the following command:

curl -XPUT 'http://localhost:9200/company/item/_mapping' -d '
{"geartrade":{"item":{"properties":{
"model":{"type":"string"},
"itemID":{"type":"integer"},
"listingDate":{"format":"dateOptionalTime","type":"date"},
"condition":{"type":"string"},
"type":{"type":"string"},
"mainImage":{"type":"string"},
"percOff":{"type":"float"},
"size":{"type":"string"},
"username":{"type":"string"},
"category":{"type":"string"},
"title":{"type":"string"},
"price":{"type":"float"},
"msrp":{"type":"float"},
"description":{"type":"string"},
"brand":{"type":"string"}}}}}'

and then when I run to command I get this message
{"ok":true,"acknowledged":true}

But then I go and run again the get mapping command and I get the
mapping unchanged

I tried updating then mapping and then refreshing, even changing the
mapping and stopping elastic search with no luck, any idea of why the
mapping does not change?

thanks a lot,

2

Hi Manuel,

Mapping is recommended to be defined at the time you are actually creating the index.
Once the index is created either explicitly or because you insert a document (and a default mapping is created for you), very few settings are available of change.

I'll suggest reviewing:
http://www.elasticsearch.org/guide/reference/api/admin-indices-create-index.html
And
http://www.elasticsearch.org/guide/reference/mapping/

~AG

3

On Thu, 2011-04-07 at 15:42 -0700, Manuel Benavides wrote:

I have the following default mapping:

{"company" : {"item" : {"properties" : {
"model" : {"type" : "string"},
"itemID" : {"type" : "string"},
... trimmed ...
"description" : { "type" : "string"},
"brand" : { "type" : "string"}}}}}

Then I go and change some values and I'll need to sort them and run
queries with ranges on some of them with the following command:

curl -XPUT 'http://localhost:9200/company/item/_mapping' -d '
{"geartrade":{"item":{"properties":{
"model":{"type":"string"},

Where does "geartrade" come from?

Your mapping syntax is wrong.

Also, you can only update a mapping if it doesn't conflict with the
existing mapping. Otherwise, you need to create a new type with the new
mapping (perhaps in a new index), and reindex your data with the new
mapping.

clint

4

I see a lot of references to "reindexing" the index in the mailing list.
But I am a bit unsure of what that really means.
For instance when you change the mappings for a type reindexing is
needed. How do I do that?

Is there an API method to call?
Do I have to delete all contents and store again?
Is reindexing the same as refresh? If not what is really refresh then?

Well, a bit of a newbie questions, but I am sure easy to answer for
anyone who knows...

5

I think you have only to push again all your documents to elastic search.

Elastic will update the mapping (or will use the new mapping if you have
provided it to ES just before) and will update each document. So you don't
have to delete anything.

Hope this helps,
David

-----Message d'origine-----
De : Kristian Jörg [mailto:krjg@devo.se]
Envoyé : vendredi 8 avril 2011 11:34
À : users@elasticsearch.com
Objet : Reindex & refresh?

I see a lot of references to "reindexing" the index in the mailing list.
But I am a bit unsure of what that really means.
For instance when you change the mappings for a type reindexing is needed.
How do I do that?

Is there an API method to call?
Do I have to delete all contents and store again?
Is reindexing the same as refresh? If not what is really refresh then?

Well, a bit of a newbie questions, but I am sure easy to answer for anyone
who knows...

6

geartrade just the name of the index I changed it on the other places for
testing and forgot that one on the console I am running it OK so the syntax
is not wrong, I think then it just shouldn't say that the mapping was
updated successfully as it does, I'll give it a try with a newly fresh
index.

On Fri, Apr 8, 2011 at 1:57 AM, Clinton Gormley clinton@iannounce.co.ukwrote:

On Thu, 2011-04-07 at 15:42 -0700, Manuel Benavides wrote:

I have the following default mapping:

{"company" : {"item" : {"properties" : {
"model" : {"type" : "string"},
"itemID" : {"type" : "string"},
... trimmed ...
"description" : { "type" : "string"},
"brand" : { "type" : "string"}}}}}

Then I go and change some values and I'll need to sort them and run
queries with ranges on some of them with the following command:

curl -XPUT 'http://localhost:9200/company/item/_mapping' -d '
{"geartrade":{"item":{"properties":{
"model":{"type":"string"},

Where does "geartrade" come from?

Your mapping syntax is wrong.

Also, you can only update a mapping if it doesn't conflict with the
existing mapping. Otherwise, you need to create a new type with the new
mapping (perhaps in a new index), and reindex your data with the new
mapping.

clint

7

Hey,

Reindexing means just that, go over what you have indexed up until now and index it back. In terms of managing indices with elasticsearch, it is recommended to reindex into a fresh index the data (possibly using updated mappings), and then hot swap it using aliases.

Getting the data to reindex can be done either by going to your where your master data is stored, and index it again (for example, from the database), or, get it from elasticsearch. Getting it from elasticsearch in an efficient manner will be possible in the upcoming version using the scan search type (already in master).

Refresh is the process of making changes done (indexing for example) visible to search. It is not related to indexing.

-shay.banon
On Friday, April 8, 2011 at 4:05 PM, David Pilato wrote:

I think you have only to push again all your documents to Elasticsearch.

Elastic will update the mapping (or will use the new mapping if you have
provided it to ES just before) and will update each document. So you don't
have to delete anything.

Hope this helps,
David

-----Message d'origine-----
De : Kristian Jörg [mailto:krjg@devo.se]
Envoyé : vendredi 8 avril 2011 11:34
À : users@elasticsearch.com
Objet : Reindex & refresh?

I see a lot of references to "reindexing" the index in the mailing list.
But I am a bit unsure of what that really means.
For instance when you change the mappings for a type reindexing is needed.
How do I do that?

Is there an API method to call?
Do I have to delete all contents and store again?
Is reindexing the same as refresh? If not what is really refresh then?

Well, a bit of a newbie questions, but I am sure easy to answer for anyone
who knows...

8

Can you provide a curl recreation? Elasticsearch Platform — Find real-time answers at scale | Elastic.
On Friday, April 8, 2011 at 5:13 PM, Manuel Antonio Benavides Salazar wrote:

geartrade just the name of the index I changed it on the other places for testing and forgot that one on the console I am running it OK so the syntax is not wrong, I think then it just shouldn't say that the mapping was updated successfully as it does, I'll give it a try with a newly fresh index.

On Fri, Apr 8, 2011 at 1:57 AM, Clinton Gormley clinton@iannounce.co.uk wrote:

On Thu, 2011-04-07 at 15:42 -0700, Manuel Benavides wrote:

I have the following default mapping:

{"company" : {"item" : {"properties" : {
"model" : {"type" : "string"},
"itemID" : {"type" : "string"},
... trimmed ...
"description" : { "type" : "string"},
"brand" : { "type" : "string"}}}}}

Then I go and change some values and I'll need to sort them and run
queries with ranges on some of them with the following command:

curl -XPUT 'http://localhost:9200/company/item/_mapping' -d '
{"geartrade":{"item":{"properties":{
"model":{"type":"string"},

Where does "geartrade" come from?

Your mapping syntax is wrong.

Also, you can only update a mapping if it doesn't conflict with the
existing mapping. Otherwise, you need to create a new type with the new
mapping (perhaps in a new index), and reindex your data with the new
mapping.

clint

9

Sure this is what I run:

curl -XPUT 'http://localhost:9200/company/item/_mapping' -d '
{"company":{
"item":{"
properties":{
"model":{"type":"string"},
"itemID":{"type":"integer"},
"listingDate":{"format":"dateOptionalTime","type":"date"},
"condition":{"type":"string"},
"type":{"type":"string"},
"mainImage":{"type":"string"},
"percOff":{"type":"float"},
"size":{"type":"string"},
"username":{"type":"string"},
"category":{"type":"string"},
"title":{"type":"string"},
"price":{"type":"float"},
"msrp":{"type":"float"},
"description":{"type":"string"},
"brand":{"type":"string"}
}
}
}}'

And when I do I get back:

{"ok":true,"acknowledged":true}

I tried with a fresh index with nothing indexed and still don't work, I just
read that you can send a mapping when you create the index so I'll try that
right now.

thanks,

On Fri, Apr 8, 2011 at 12:27 PM, Shay Banon shay.banon@elasticsearch.comwrote:

Can you provide a curl recreation? Elasticsearch Platform — Find real-time answers at scale | Elastic.

On Friday, April 8, 2011 at 5:13 PM, Manuel Antonio Benavides Salazar
wrote:

geartrade just the name of the index I changed it on the other places for
testing and forgot that one on the console I am running it OK so the syntax
is not wrong, I think then it just shouldn't say that the mapping was
updated successfully as it does, I'll give it a try with a newly fresh
index.

On Fri, Apr 8, 2011 at 1:57 AM, Clinton Gormley clinton@iannounce.co.ukwrote:

On Thu, 2011-04-07 at 15:42 -0700, Manuel Benavides wrote:

I have the following default mapping:

{"company" : {"item" : {"properties" : {
"model" : {"type" : "string"},
"itemID" : {"type" : "string"},
... trimmed ...
"description" : { "type" : "string"},
"brand" : { "type" : "string"}}}}}

Then I go and change some values and I'll need to sort them and run
queries with ranges on some of them with the following command:

curl -XPUT 'http://localhost:9200/company/item/_mapping' -d '
{"geartrade":{"item":{"properties":{
"model":{"type":"string"},

Where does "geartrade" come from?

Your mapping syntax is wrong.

Also, you can only update a mapping if it doesn't conflict with the
existing mapping. Otherwise, you need to create a new type with the new
mapping (perhaps in a new index), and reindex your data with the new
mapping.

clint

10

Its because the top level field is the index name, which it shouldn't be. The top level field should be just the type name, without the index name. I understand (I think) where the confusion is coming from, because of the GET Mapping API response when asking for a specific type mapping in a single index, thats why I added this to master:

Issues · elastic/elasticsearch · GitHub.

On Friday, April 8, 2011 at 11:37 PM, Manuel Antonio Benavides Salazar wrote:

Sure this is what I run:

curl -XPUT 'http://localhost:9200/company/item/_mapping' -d '
{"company":{
"item":{"
properties":{
"model":{"type":"string"},
"itemID":{"type":"integer"},
"listingDate":{"format":"dateOptionalTime","type":"date"},
"condition":{"type":"string"},
"type":{"type":"string"},
"mainImage":{"type":"string"},
"percOff":{"type":"float"},
"size":{"type":"string"},
"username":{"type":"string"},
"category":{"type":"string"},
"title":{"type":"string"},
"price":{"type":"float"},
"msrp":{"type":"float"},
"description":{"type":"string"},
"brand":{"type":"string"}
}
}
}}'

And when I do I get back:

{"ok":true,"acknowledged":true}

I tried with a fresh index with nothing indexed and still don't work, I just read that you can send a mapping when you create the index so I'll try that right now.

thanks,

On Fri, Apr 8, 2011 at 12:27 PM, Shay Banon shay.banon@elasticsearch.com wrote:

Can you provide a curl recreation? Elasticsearch Platform — Find real-time answers at scale | Elastic.
On Friday, April 8, 2011 at 5:13 PM, Manuel Antonio Benavides Salazar wrote:

geartrade just the name of the index I changed it on the other places for testing and forgot that one on the console I am running it OK so the syntax is not wrong, I think then it just shouldn't say that the mapping was updated successfully as it does, I'll give it a try with a newly fresh index.

On Fri, Apr 8, 2011 at 1:57 AM, Clinton Gormley clinton@iannounce.co.uk wrote:

On Thu, 2011-04-07 at 15:42 -0700, Manuel Benavides wrote:

I have the following default mapping:

{"company" : {"item" : {"properties" : {
"model" : {"type" : "string"},
"itemID" : {"type" : "string"},
... trimmed ...
"description" : { "type" : "string"},
"brand" : { "type" : "string"}}}}}

Then I go and change some values and I'll need to sort them and run
queries with ranges on some of them with the following command:

curl -XPUT 'http://localhost:9200/company/item/_mapping' -d '
{"geartrade":{"item":{"properties":{
"model":{"type":"string"},

Where does "geartrade" come from?

Your mapping syntax is wrong.

Also, you can only update a mapping if it doesn't conflict with the
existing mapping. Otherwise, you need to create a new type with the new
mapping (perhaps in a new index), and reindex your data with the new
mapping.

clint

11

Great just tried it like this and totally worked, yeah is a bit confusing
but really makes sense thanks a ton.

curl -XPUT 'http://localhost:9200/geartrade/_mapping' -d
'{"item":{"properties":{"model":{"type":"string"},"itemID":{"type":"integer"},"listingDate":{"format":"dateOptionalTime","type":"date"},"condition":{"type":"string"},"type":{"type":"string"},"mainImage":{"type":"string"},"percOff":{"type":"float"},"size":{"type":"string"},"username":{"type":"string"},"category":{"type":"string"},"title":{"type":"string"},"price":{"type":"float"},"msrp":{"type":"float"},"description":{"type":"string"},"brand":{"type":"string"}}}}'

On Fri, Apr 8, 2011 at 2:52 PM, Shay Banon shay.banon@elasticsearch.comwrote:

Its because the top level field is the index name, which it shouldn't be.
The top level field should be just the type name, without the index name. I
understand (I think) where the confusion is coming from, because of the GET
Mapping API response when asking for a specific type mapping in a single
index, thats why I added this to master:

Issues · elastic/elasticsearch · GitHub.

On Friday, April 8, 2011 at 11:37 PM, Manuel Antonio Benavides Salazar
wrote:

Sure this is what I run:

curl -XPUT 'http://localhost:9200/company/item/_mapping' -d '
{"company":{
"item":{"
properties":{
"model":{"type":"string"},
"itemID":{"type":"integer"},
"listingDate":{"format":"dateOptionalTime","type":"date"},
"condition":{"type":"string"},
"type":{"type":"string"},
"mainImage":{"type":"string"},
"percOff":{"type":"float"},
"size":{"type":"string"},
"username":{"type":"string"},
"category":{"type":"string"},
"title":{"type":"string"},
"price":{"type":"float"},
"msrp":{"type":"float"},
"description":{"type":"string"},
"brand":{"type":"string"}
}
}
}}'

And when I do I get back:

{"ok":true,"acknowledged":true}

I tried with a fresh index with nothing indexed and still don't work, I
just read that you can send a mapping when you create the index so I'll try
that right now.

thanks,

On Fri, Apr 8, 2011 at 12:27 PM, Shay Banon shay.banon@elasticsearch.comwrote:

Can you provide a curl recreation? Elasticsearch Platform — Find real-time answers at scale | Elastic.

On Friday, April 8, 2011 at 5:13 PM, Manuel Antonio Benavides Salazar
wrote:

geartrade just the name of the index I changed it on the other places for
testing and forgot that one on the console I am running it OK so the syntax
is not wrong, I think then it just shouldn't say that the mapping was
updated successfully as it does, I'll give it a try with a newly fresh
index.

On Fri, Apr 8, 2011 at 1:57 AM, Clinton Gormley clinton@iannounce.co.ukwrote:

On Thu, 2011-04-07 at 15:42 -0700, Manuel Benavides wrote:

I have the following default mapping:

{"company" : {"item" : {"properties" : {
"model" : {"type" : "string"},
"itemID" : {"type" : "string"},
... trimmed ...
"description" : { "type" : "string"},
"brand" : { "type" : "string"}}}}}

Then I go and change some values and I'll need to sort them and run
queries with ranges on some of them with the following command:

curl -XPUT 'http://localhost:9200/company/item/_mapping' -d '
{"geartrade":{"item":{"properties":{
"model":{"type":"string"},

Where does "geartrade" come from?

Your mapping syntax is wrong.

Also, you can only update a mapping if it doesn't conflict with the
existing mapping. Otherwise, you need to create a new type with the new
mapping (perhaps in a new index), and reindex your data with the new
mapping.

clint

12

Thanks a lot Shay's suggestion totally worked for me and thanks for the
recommendation I'll do it that way as the mapping automatically created
doesn't match all elements as I need them.

cheers,

On Thu, Apr 7, 2011 at 4:55 PM, AGuereca aguereca@gmail.com wrote:

Hi Manuel,

Mapping is recommended to be defined at the time you are actually creating
the index.
Once the index is created either explicitly or because you insert a
document
(and a default mapping is created for you), very few settings are available
of change.

I'll suggest reviewing:

Elasticsearch Platform — Find real-time answers at scale | Elastic

Elasticsearch Platform — Find real-time answers at scale | Elastic
And
Elasticsearch Platform — Find real-time answers at scale | Elastic
Elasticsearch Platform — Find real-time answers at scale | Elastic

~AG

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/changing-default-mapping-tp2792694p2792755.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

13 
Ok!




Thanks for clearing that out. Now I understand what aliases can be
used for as well. :)




Btw, as I an afterthought, would it not be nice to have a method in
ES to do the reindexing for you. I.e internally create a new tmp
index, reindex with the scan search type and efter all is done
return to you a complete reindexed index with the same name as
before. If possible it could even be done in production and
seamlessly switching the old index for the new when the process is
done. The API method for this could simply be "reindex".




Just a thought, as reindexing seems to be a fairly common procedure
while doing configuration changes.




/Kristian




Shay Banon skrev 2011-04-08 20:23:
<blockquote cite="mid:371DC248F11A4770AD25BCF9B1B2B8B1@elasticsearch.com" type="cite">Hey,

  Reindexing means just that, go over what you have
indexed up until now and index it back. In terms of managing
indices with elasticsearch, it is recommended to reindex
into a fresh index the data (possibly using updated
mappings), and then hot swap it using aliases.

  Getting the data to reindex can be done either by
going to your where your master data is stored, and index it
again (for example, from the database), or, get it from
elasticsearch. Getting it from elasticsearch in an efficient
manner will be possible in the upcoming version using the
scan search type (already in master).

  Refresh is the process of making changes done
(indexing for example) visible to search. It is not related
to indexing.

-shay.banon

On Friday, April 8, 2011 at 4:05 PM, David Pilato wrote:

I think you have only to push again all your documents to elastic search. 
              Elastic will update the mapping (or will use the new
              mapping if you have


              provided it to ES just before) and will update each
              document. So you don't


              have to delete anything.




              Hope this helps,


              David




              -----Message d'origine-----


              De : Kristian Jörg [<a moz-do-not-send="true" href="mailto:krjg@devo.se">mailto:krjg@devo.se</a>]
              


              Envoyé : vendredi 8 avril 2011 11:34


              À : <a moz-do-not-send="true" href="mailto:users@elasticsearch.com">users@elasticsearch.com</a>


              Objet : Reindex &amp; refresh?




              I see a lot of references to "reindexing" the index in
              the mailing list. 


              But I am a bit unsure of what that really means.


              For instance when you change the mappings for a type
              reindexing is needed.


              How do I do that?




              Is there an API method to call?


              Do I have to delete all contents and store again?


              Is reindexing the same as refresh? If not what is
              really refresh then?




              Well, a bit of a newbie questions, but I am sure easy
              to answer for anyone


              who knows...

-- 
Med vänlig hälsning
Kristian Jörg

Devo IT AB
Tel: 054 - 22 14 58, 0709 - 15 83 42
E-post: kristian.jorg@devo.se
Webb: http://www.devo.se
14

A reindex API is certainly possible, the main effort was getting the scan API first.
On Monday, April 11, 2011 at 9:39 AM, Kristian Jörg wrote:

Ok!

Thanks for clearing that out. Now I understand what aliases can be used for as well. :slight_smile:

Btw, as I an afterthought, would it not be nice to have a method in ES to do the reindexing for you. I.e internally create a new tmp index, reindex with the scan search type and efter all is done return to you a complete reindexed index with the same name as before. If possible it could even be done in production and seamlessly switching the old index for the new when the process is done. The API method for this could simply be "reindex".

Just a thought, as reindexing seems to be a fairly common procedure while doing configuration changes.

/Kristian

Shay Banon skrev 2011-04-08 20:23:

Hey,

Reindexing means just that, go over what you have indexed up until now and index it back. In terms of managing indices with elasticsearch, it is recommended to reindex into a fresh index the data (possibly using updated mappings), and then hot swap it using aliases.

Getting the data to reindex can be done either by going to your where your master data is stored, and index it again (for example, from the database), or, get it from elasticsearch. Getting it from elasticsearch in an efficient manner will be possible in the upcoming version using the scan search type (already in master).

Refresh is the process of making changes done (indexing for example) visible to search. It is not related to indexing.

-shay.banon
On Friday, April 8, 2011 at 4:05 PM, David Pilato wrote:

I think you have only to push again all your documents to Elasticsearch.

Elastic will update the mapping (or will use the new mapping if you have
provided it to ES just before) and will update each document. So you don't
have to delete anything.

Hope this helps,
David

-----Message d'origine-----
De : Kristian Jörg [mailto:krjg@devo.se]
Envoyé : vendredi 8 avril 2011 11:34
À : users@elasticsearch.com
Objet : Reindex & refresh?

I see a lot of references to "reindexing" the index in the mailing list.
But I am a bit unsure of what that really means.
For instance when you change the mappings for a type reindexing is needed.
How do I do that?

Is there an API method to call?
Do I have to delete all contents and store again?
Is reindexing the same as refresh? If not what is really refresh then?

Well, a bit of a newbie questions, but I am sure easy to answer for anyone
who knows...

-- Med vänlig hälsning Kristian Jörg Devo IT AB Tel: 054 - 22 14 58, 0709 - 15 83 42 E-post: kristian.jorg@devo.se Webb: http://www.devo.se