Changing simple type mapping

Elastic Stack Elasticsearch
1

Hi all,

I need to change the mapping on one of my fields. It is currently an
integer and I need it to be a string.

Ideally, I would be able to just change the type. But the documentation
says that I cannot do that (unless that has changed in the last year, I
think all the articles I read were at least a year old).

The next best suggestion I liked was changing the mapping to be a multi
field. So I did the following:

curl -XPUT 'http://localhost:9200/myIndex/myType/_mapping' -d
'{"myType":{"properties":{"copyrightYear":{"type":"multi_field","fields":{"copyrightYear":{"type":"integer"},"textual":{"type":"string"}}}}}}'

But the result is:

curl -XGET 'http://localhost:9200/myIndex/_mapping/myType'
{"myIndex":{"mappings":{"myType":{"index_analyzer":"indexAnalyzer","search_analyzer":"searchAnalyzer","_boost":{"null_value":1.0},"properties":{"isInitialized":{"type":"string"},,"copyrightYear":{"type":"integer","fields":{"copyrightYear":{"type":"integer"},"textual":{"type":"string"}}}

The issue is that when I try to add a document that has copyrightYear =
'2013/2014' for example, I get an error:

Error in one or more bulk request actions:

index: /myIndex/myType/73148865 caused MapperParsingException[failed to
parse [copyright.copyrightYear]]; nested: NumberFormatException[For input
string: "2013/2014"];

Note that the adding uses the Elastica client but the error is due to the
mapping. Is there a better approach to this problem? I don't really want to
reindex all the things because I have about 22 million records.

Thanks!

Brian Lamb

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8adc6f39-85bd-4610-b354-8457b77b8cfd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

You need to reindex or you need to add a new field IMHO.

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 23 mai 2014 à 18:09, Brian Lamb brian.lamb@researchsquare.com a écrit :

Hi all,

I need to change the mapping on one of my fields. It is currently an integer and I need it to be a string.

Ideally, I would be able to just change the type. But the documentation says that I cannot do that (unless that has changed in the last year, I think all the articles I read were at least a year old).

The next best suggestion I liked was changing the mapping to be a multi field. So I did the following:

curl -XPUT 'http://localhost:9200/myIndex/myType/_mapping' -d '{"myType":{"properties":{"copyrightYear":{"type":"multi_field","fields":{"copyrightYear":{"type":"integer"},"textual":{"type":"string"}}}}}}'

But the result is:

curl -XGET 'http://localhost:9200/myIndex/_mapping/myType'
{"myIndex":{"mappings":{"myType":{"index_analyzer":"indexAnalyzer","search_analyzer":"searchAnalyzer","_boost":{"null_value":1.0},"properties":{"isInitialized":{"type":"string"},,"copyrightYear":{"type":"integer","fields":{"copyrightYear":{"type":"integer"},"textual":{"type":"string"}}}

The issue is that when I try to add a document that has copyrightYear = '2013/2014' for example, I get an error:

Error in one or more bulk request actions:

index: /myIndex/myType/73148865 caused MapperParsingException[failed to parse [copyright.copyrightYear]]; nested: NumberFormatException[For input string: "2013/2014"];

Note that the adding uses the Elastica client but the error is due to the mapping. Is there a better approach to this problem? I don't really want to reindex all the things because I have about 22 million records.

Thanks!

Brian Lamb

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8adc6f39-85bd-4610-b354-8457b77b8cfd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/D6F67166-2DBD-419F-9F7D-3037C19B9428%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.

3

Hi Brian

What you can do is to update the mapping of the numeric field to {
ignore_malformed: true }, then it will just ignore the bad data, but still
use the multi-field:

PUT /my_index/my_type/1
{"number": 123}

PUT /my_index/_mapping/my_type
{
  "properties": {
    "number": {
      "type": "long",
      "ignore_malformed": true,
      "fields": {
        "string": {
          "type": "string"
        }
      }
    }
  }
}

PUT /my_index/my_type/1
{"number": "x"}

GET /_search
{
  "query": {
    "match": {
      "number.string": "x"
    }
  }
}

On 23 May 2014 18:09, Brian Lamb brian.lamb@researchsquare.com wrote:

Hi all,

I need to change the mapping on one of my fields. It is currently an
integer and I need it to be a string.

Ideally, I would be able to just change the type. But the documentation
says that I cannot do that (unless that has changed in the last year, I
think all the articles I read were at least a year old).

The next best suggestion I liked was changing the mapping to be a multi
field. So I did the following:

curl -XPUT 'http://localhost:9200/myIndex/myType/_mapping' -d
'{"myType":{"properties":{"copyrightYear":{"type":"multi_field","fields":{"copyrightYear":{"type":"integer"},"textual":{"type":"string"}}}}}}'

But the result is:

curl -XGET 'http://localhost:9200/myIndex/_mapping/myType'

{"myIndex":{"mappings":{"myType":{"index_analyzer":"indexAnalyzer","search_analyzer":"searchAnalyzer","_boost":{"null_value":1.0},"properties":{"isInitialized":{"type":"string"},,"copyrightYear":{"type":"integer","fields":{"copyrightYear":{"type":"integer"},"textual":{"type":"string"}}}

The issue is that when I try to add a document that has copyrightYear =
'2013/2014' for example, I get an error:

Error in one or more bulk request actions:

index: /myIndex/myType/73148865 caused MapperParsingException[failed to
parse [copyright.copyrightYear]]; nested: NumberFormatException[For input
string: "2013/2014"];

Note that the adding uses the Elastica client but the error is due to the
mapping. Is there a better approach to this problem? I don't really want to
reindex all the things because I have about 22 million records.

Thanks!

Brian Lamb

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/8adc6f39-85bd-4610-b354-8457b77b8cfd%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/8adc6f39-85bd-4610-b354-8457b77b8cfd%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPt3XKQnp6C6RrHC7FZG20s-Y%3DGp3zTrKZdzqCcsDjVDNBshKg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

4

w00t! Sounds like I missed that option.

Thanks Clint!

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr

Le 23 mai 2014 à 19:06:50, Clinton Gormley (clint@traveljury.com) a écrit:

Hi Brian

What you can do is to update the mapping of the numeric field to { ignore_malformed: true }, then it will just ignore the bad data, but still use the multi-field:

PUT /my_index/my_type/1
{"number": 123}

PUT /my_index/_mapping/my_type
{
  "properties": {
    "number": {
      "type": "long",
      "ignore_malformed": true,
      "fields": {
        "string": {
          "type": "string"
        }
      }
    }
  }
}

PUT /my_index/my_type/1
{"number": "x"}

GET /_search
{
  "query": {
    "match": {
      "number.string": "x"
    }
  }
}

On 23 May 2014 18:09, Brian Lamb brian.lamb@researchsquare.com wrote:
Hi all,

I need to change the mapping on one of my fields. It is currently an integer and I need it to be a string.

Ideally, I would be able to just change the type. But the documentation says that I cannot do that (unless that has changed in the last year, I think all the articles I read were at least a year old).

The next best suggestion I liked was changing the mapping to be a multi field. So I did the following:

curl -XPUT 'http://localhost:9200/myIndex/myType/_mapping' -d '{"myType":{"properties":{"copyrightYear":{"type":"multi_field","fields":{"copyrightYear":{"type":"integer"},"textual":{"type":"string"}}}}}}'

But the result is:

curl -XGET 'http://localhost:9200/myIndex/_mapping/myType'
{"myIndex":{"mappings":{"myType":{"index_analyzer":"indexAnalyzer","search_analyzer":"searchAnalyzer","_boost":{"null_value":1.0},"properties":{"isInitialized":{"type":"string"},,"copyrightYear":{"type":"integer","fields":{"copyrightYear":{"type":"integer"},"textual":{"type":"string"}}}

The issue is that when I try to add a document that has copyrightYear = '2013/2014' for example, I get an error:

Error in one or more bulk request actions:

index: /myIndex/myType/73148865 caused MapperParsingException[failed to parse [copyright.copyrightYear]]; nested: NumberFormatException[For input string: "2013/2014"];

Note that the adding uses the Elastica client but the error is due to the mapping. Is there a better approach to this problem? I don't really want to reindex all the things because I have about 22 million records.

Thanks!

Brian Lamb

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8adc6f39-85bd-4610-b354-8457b77b8cfd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPt3XKQnp6C6RrHC7FZG20s-Y%3DGp3zTrKZdzqCcsDjVDNBshKg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.537f8098.19495cff.1e56%40MacBook-Air-de-David.local.
For more options, visit https://groups.google.com/d/optout.

5

Hi Clinton,

Thank you for your suggestion. What will that do for the existing data?

Will I still be able to store copyrightYear as either a number or a string?

Thanks,

Brian Lamb

On Friday, May 23, 2014 1:09:46 PM UTC-4, David Pilato wrote:

w00t! Sounds like I missed that option.

Thanks Clint!

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr

Le 23 mai 2014 à 19:06:50, Clinton Gormley (cl...@traveljury.com<javascript:>)
a écrit:

Hi Brian

What you can do is to update the mapping of the numeric field to {
ignore_malformed: true }, then it will just ignore the bad data, but still
use the multi-field:

 PUT /my_index/my_type/1
{"number": 123}

PUT /my_index/_mapping/my_type
{
  "properties": {
    "number": {
      "type": "long",
      "ignore_malformed": true,
      "fields": {
        "string": {
          "type": "string"
        }
      }
    }
  }
}

PUT /my_index/my_type/1
{"number": "x"}

GET /_search
{
  "query": {
    "match": {
      "number.string": "x"
    }
  }
}

On 23 May 2014 18:09, Brian Lamb <brian...@researchsquare.com<javascript:>

wrote:

Hi all,

I need to change the mapping on one of my fields. It is currently an
integer and I need it to be a string.

Ideally, I would be able to just change the type. But the documentation
says that I cannot do that (unless that has changed in the last year, I
think all the articles I read were at least a year old).

The next best suggestion I liked was changing the mapping to be a multi
field. So I did the following:

curl -XPUT 'http://localhost:9200/myIndex/myType/_mapping' -d
'{"myType":{"properties":{"copyrightYear":{"type":"multi_field","fields":{"copyrightYear":{"type":"integer"},"textual":{"type":"string"}}}}}}'

But the result is:

curl -XGET 'http://localhost:9200/myIndex/_mapping/myType'

{"myIndex":{"mappings":{"myType":{"index_analyzer":"indexAnalyzer","search_analyzer":"searchAnalyzer","_boost":{"null_value":1.0},"properties":{"isInitialized":{"type":"string"},,"copyrightYear":{"type":"integer","fields":{"copyrightYear":{"type":"integer"},"textual":{"type":"string"}}}

The issue is that when I try to add a document that has copyrightYear =
'2013/2014' for example, I get an error:

Error in one or more bulk request actions:

index: /myIndex/myType/73148865 caused MapperParsingException[failed to
parse [copyright.copyrightYear]]; nested: NumberFormatException[For input
string: "2013/2014"];

Note that the adding uses the Elastica client but the error is due to the
mapping. Is there a better approach to this problem? I don't really want to
reindex all the things because I have about 22 million records.

Thanks!

Brian Lamb

You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/8adc6f39-85bd-4610-b354-8457b77b8cfd%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/8adc6f39-85bd-4610-b354-8457b77b8cfd%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAPt3XKQnp6C6RrHC7FZG20s-Y%3DGp3zTrKZdzqCcsDjVDNBshKg%40mail.gmail.comhttps://groups.google.com/d/msgid/elasticsearch/CAPt3XKQnp6C6RrHC7FZG20s-Y%3DGp3zTrKZdzqCcsDjVDNBshKg%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/44b6f6a9-5ac1-4f74-9eb5-66c2ec662022%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

6

Hi Clinton,

Thank you for your suggestion. What will that do for the existing data?

Will I still be able to store copyrightYear as either a number or a string?

Thanks,

Brian Lamb

On Friday, May 23, 2014 1:09:46 PM UTC-4, David Pilato wrote:

w00t! Sounds like I missed that option.

Thanks Clint!

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr

Le 23 mai 2014 à 19:06:50, Clinton Gormley (cl...@traveljury.com<javascript:>)
a écrit:

Hi Brian

What you can do is to update the mapping of the numeric field to {
ignore_malformed: true }, then it will just ignore the bad data, but still
use the multi-field:

 PUT /my_index/my_type/1
{"number": 123}

PUT /my_index/_mapping/my_type
{
  "properties": {
    "number": {
      "type": "long",
      "ignore_malformed": true,
      "fields": {
        "string": {
          "type": "string"
        }
      }
    }
  }
}

PUT /my_index/my_type/1
{"number": "x"}

GET /_search
{
  "query": {
    "match": {
      "number.string": "x"
    }
  }
}

On 23 May 2014 18:09, Brian Lamb <brian...@researchsquare.com<javascript:>

wrote:

Hi all,

I need to change the mapping on one of my fields. It is currently an
integer and I need it to be a string.

Ideally, I would be able to just change the type. But the documentation
says that I cannot do that (unless that has changed in the last year, I
think all the articles I read were at least a year old).

The next best suggestion I liked was changing the mapping to be a multi
field. So I did the following:

curl -XPUT 'http://localhost:9200/myIndex/myType/_mapping' -d
'{"myType":{"properties":{"copyrightYear":{"type":"multi_field","fields":{"copyrightYear":{"type":"integer"},"textual":{"type":"string"}}}}}}'

But the result is:

curl -XGET 'http://localhost:9200/myIndex/_mapping/myType'

{"myIndex":{"mappings":{"myType":{"index_analyzer":"indexAnalyzer","search_analyzer":"searchAnalyzer","_boost":{"null_value":1.0},"properties":{"isInitialized":{"type":"string"},,"copyrightYear":{"type":"integer","fields":{"copyrightYear":{"type":"integer"},"textual":{"type":"string"}}}

The issue is that when I try to add a document that has copyrightYear =
'2013/2014' for example, I get an error:

Error in one or more bulk request actions:

index: /myIndex/myType/73148865 caused MapperParsingException[failed to
parse [copyright.copyrightYear]]; nested: NumberFormatException[For input
string: "2013/2014"];

Note that the adding uses the Elastica client but the error is due to the
mapping. Is there a better approach to this problem? I don't really want to
reindex all the things because I have about 22 million records.

Thanks!

Brian Lamb

You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/8adc6f39-85bd-4610-b354-8457b77b8cfd%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/8adc6f39-85bd-4610-b354-8457b77b8cfd%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAPt3XKQnp6C6RrHC7FZG20s-Y%3DGp3zTrKZdzqCcsDjVDNBshKg%40mail.gmail.comhttps://groups.google.com/d/msgid/elasticsearch/CAPt3XKQnp6C6RrHC7FZG20s-Y%3DGp3zTrKZdzqCcsDjVDNBshKg%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/aac85e1c-7f89-4a1e-b293-490a67e9f43b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

7

Thank you for your suggestion. What will that do for the existing data?

Will I still be able to store copyrightYear as either a number or a string?

It won't change any existing data. However, for data you index in the
future it will index either a number and a string, or (if it can't coerce
the value into a number), just a string, eg: if you index these two docs:

{ "copyrightYear": "2010"}
{ "copyrightYear": "2011/2012"}

then "copyrightYear" will contain 2010, and "copyrightYear.textual" will
contain "2010", "2011", "2012"

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPt3XKRr5CLjRL1D_FtHbewaJWz2%3DoNC5j403zwwtvWNk28eqA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.