Elastic 5 search result returns incorrect value of custom "id" field

Elastic Stack Elasticsearch
1

I recently upgrade to Elastic 5.1.1 and my unit test starts to fail. The context of the test is to put a document in the index and search it. simple. The search result does correctly return the record, however the custom "id" field has incorrect value that doesn't belong to any other document. btw, there is only 1 record in the index.

Here are the details:

This is the document that is indexed to be search. Notice the "id" is 202285124

{
  "_index": "908c08cd93884c86b560848900a8968f",
  "_type": "contact",
  "_id": "202285124",
  "_version": 1,
  "_routing": "809794",
  "found": true,
  "_source": {
    "salutation": "ed3c65cec1184a89b71fb68b6d3dd294",
    "firstName": "3cb659bcac7d4bd4b27c64125671fd2a",
    "lastName": "3cb659bc-ac7d-4bd4-b27c-64125671fd2a",
    "background": "3a74e279-bcb5-4e0e-bb63-a9254d345161",
    "emails": [],
    "phones": [],
    "addresses": [],
    "websites": [],
    "tags": [],
    "customFields": [],
    "dateToRemembers": [],
    "visibleTo": "OWNER",
    "instanceId": 809794,
    "id": 202285124,
    "typeName": "contact",
    "typeLabel": "contact",
    "title": "ed3c65cec1184a89b71fb68b6d3dd294 3cb659bcac7d4bd4b27c64125671fd2a 3cb659bc-ac7d-4bd4-b27c-64125671fd2a",
    "autocomplete_title": "ed3c65cec1184a89b71fb68b6d3dd294 3cb659bcac7d4bd4b27c64125671fd2a 3cb659bc-ac7d-4bd4-b27c-64125671fd2a",
    "shingle_title": "ed3c65cec1184a89b71fb68b6d3dd294 3cb659bcac7d4bd4b27c64125671fd2a 3cb659bc-ac7d-4bd4-b27c-64125671fd2a",
    "dateCreated": "2017-02-14T19:49:26.303-08:00",
    "dateUpdated": "2017-02-14T19:49:26.303-08:00",
    "ownerUserId": 1462234
  }
}

This is the search query that returns the "id" by "instanceId".

{
	"stored_fields": ["id"],
	"query": {
		"term": {
			"instanceId": 809794
		}
	}
}

This is the result. As you can see, the value of the "id" is 202285120 instead of 202285124.

{
  "took": 7,
  "timed_out": false,
  "_shards": {
    "total": 11,
    "successful": 11,
    "failed": 0
  },
  "hits": {
    "total": 1,
    "max_score": 1,
    "hits": [
      {
        "_index": "908c08cd93884c86b560848900a8968f",
        "_type": "contact",
        "_id": "202285124",
        "_score": 1,
        "_routing": "809794",
        "fields": {
          "id": [
            202285120
          ]
        }
      }
    ]
  }
}

As I mentioned, I don't have this issue with Elastic 2.3/2.4. Please advise.

2

I think it is a mapping issue due to the fact that your id field is mapped as a float. Floats can only represent integers accurately up to 16777216, higher absolute values get rounded. If you use this field for ranges, you should map it as a long but if you actually use it as an id field and only look up single values, then it would be best mapped as a keyword.

3

I think you might be right about it. However this field has attribute hint when schema is generated with [Number(Store = true)]. Is there any setting I should set to prevent the schema set the data type as long instead of float? I will do some more testings on this. Thanks again.

4

Store = true needs to be set because of the new introduced stored_field. But the Store = true seems to interfere with NumberAttribute.Scaling_Factor, one will not set if other is set. It is a bug in my opinion. The workaround I have is to use Source instead of Stored_Field in query search.

5

I don't think Store=true is related to the problem. The problem is really how the field is mapped. I never heard aboutNumber(Store = true) and NumberAttribute.Scaling_Factor, are you using a tool on top of Elasticsearch that manages your mappings?

6

Yes, I am using the NEST APIs from github (https://github.com/elastic/elasticsearch-net) to generate schema by using automap feature with the Attribute hints the define the field properties. Every Attribute has a Store property and for NumberAttribute the ScalingFactor will force the data type to long instead of default to float. Again, this all works in 2.4 without store or scaling factor. That is why I suggests 5.0 either changes the behavior or broke it.

Nest NumberAttribute

Number Data Type - Scaling Factor Property
https://www.elastic.co/guide/en/elasticsearch/reference/current/number.html

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.