Number format Exception For string type

Elastic Stack Elasticsearch
1

I have a mapping like this

"users": {
"user_profiles": {
"properties": {
"score" : {
"type": "string"
}
}
}
}

I can get the score as integer as well as "NA" so i mapped the type as string but while posting data to the index i am getting Number Format Exception.

while checking my log file i am getting this errors:

[2016-08-29 15:19:01] elasticlog.WARNING: Response ["{"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"failed to parse [score]"}],"type":"mapper_parsing_exception","reason":"failed to parse [score]","caused_by":{"type":"number_format_exception","reason":"For input string: \"NH\""}},"status":400}"] []

2

Hi @parithiban,

any chance that you have defined this mapping a little while after you have created the index? Because in that case, Elasticsearch has probably derived a mapping automatically (we call that "dynamic mapping").

To me it looks like the field score is defined as numeric field as you can see from the error description.

In case this was just a playground index, delete it and create the mapping when you create the index.

If this is an index with production data that you cannot afford to wipe, you need to create a new index including the mapping that you want and use the reindex API to index the data again in the new index.

Daniel

3

Thanks for the reply @danielmitterdorfer .We did the migration by first running the mapping and then only indexing data. The score index was already in the mapping.

My issue is that the first 100 records contain the score as integer it got indexed successfully and then after that the score was "NA" at this time only i am getting issue

4

Hi @parithiban,

ah, now I understand. However, it appears to me that your mapping is different than what you've shown above. You can check the mapping with GET /YOUR_INDEX_NAME_HERE/_mapping. Check the response for the score field. It will most likely be a numeric field.

Daniel

5

I have same issue recently. i have production index mapping where i defined "auid" as "string", then later on i found it should be "integer" so i created a new index with mapping "auid" as integer, then using reindex API to re-index data from old index to the newindex, but I am getting the exception below??

  "failures" : [
    {
      "index" : "newindex",
      "type" : "author",
      "id" : "123",
      "cause" : {
        "type" : "mapper_parsing_exception",
        "reason" : "failed to parse [auid]",
        "caused_by" : {
          "type" : "number_format_exception",
          "reason" : "For input string: \"1111\""
        }
      },
      "status" : 400
    },
	```
6

Hi,

did you create the new index before running the reindex? Please follow the example below. This worked fine for me on Elasticsearch 2.4.5 (as your talking about "string" I assumed you are on 2.x; in 5.x "string" type has been split into "text" and "keyword").

Let's start fresh and delete all involved indices for this example:

DELETE /old_books
DELETE /new_books

Then create the old index and add a document to it.

PUT /old_books
{
   "mappings": {
      "author": {
         "properties": {
            "name": {
               "type": "string"
            },
            "auid": {
                "type": "string"
            }
         }
      }
   }
}

PUT /old_books/author/1
{
    "name": "Mark Twain",
    "auid": "42"
}

Now we realize we made a mistake and create the new index where we correct the mapping of the auid field to integer:

PUT /new_books
{
   "mappings": {
      "author": {
         "properties": {
            "name": {
               "type": "string"
            },
            "auid": {
                "type": "integer"
            }
         }
      }
   }
}

and let the reindex API do its job:

POST /_reindex
{
  "source": {
    "index": "old_books"
  },
  "dest": {
    "index": "new_books"
  }
}

This worked fine for me. Here is the response:

{
   "took": 56,
   "timed_out": false,
   "total": 1,
   "updated": 0,
   "created": 1,
   "batches": 1,
   "version_conflicts": 0,
   "noops": 0,
   "retries": 0,
   "throttled_millis": 0,
   "requests_per_second": "unlimited",
   "throttled_until_millis": 0,
   "failures": []
}

Note, although Elasticsearch indexes the field now as integer, it is still a string in _source:

GET /new_books/author/_search
{
    "query": {
        "match_all": {}
    }
}

shows:

{
  ...
      "hits": [
         {
            "_index": "new_books",
            "_type": "author",
            "_id": "1",
            "_score": 1,
            "_source": {
               "name": "Mark Twain",
               "auid": "42"
            }
         }
      ]
...

If you do not want that you can use an inline script to convert the source as well during reindexing. Remember that inline scripts are a security risk and that you should rather use stored scripts for this. If you still want to use inline scripts then you need to (temporarily) enable them by setting script.inline: true in config/elasticsearch.yml and restart your cluster:

POST /_reindex
{
  "source": {
    "index": "old_books"
  },
  "dest": {
    "index": "new_books"
  },
"script": {
    "inline": "ctx._source.auid = Integer.valueOf(ctx._source.auid)"
  }
}

This will significantly slow-down the reindexing process though. Also check how you can reindex without downtime.

Daniel

7

many thanks Danial for your thorough explanation.

actually am using AWS ES 5.1. i did not change string to text bc i did not get any exception. though i just changed all "string" fields to "text". i followed same instructions as follows, but still getting same error

  1. old index "old_books" mapping contains "auid" as "string"
  2. create new index with new mapping by setting "auid" as "integer"
  3. re-index

POST /_reindex
{
"source": {
"index": "old_books"
},
"dest": {
"index": "new_books"
}
}

Result:

{
"index" : "new_books",
"type" : "author",
"id" : "123",
"cause" : {
"type" : "mapper_parsing_exception",
"reason" : "failed to parse [audoc.auid]",
"caused_by" : {
"type" : "number_format_exception",
"reason" : "For input string: "1111""
}
},
"status" : 400
}
]
}

8

well, i figured out the issue was due to some "auid" has value > max length of integer datatype "2^31-1". by changing datatype to long, reindex went through with no issues.