# \_id value is getting changed in elastic

**URL:** <https://discuss.elastic.co/t/id-value-is-getting-changed-in-elastic/268905>\
**Category:** Elasticsearch\
**Created:** [March 31, 2021, 12:41pm UTC](https://discuss.elastic.co/t/id-value-is-getting-changed-in-elastic/268905 "2021-03-31T12:41:26Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![gbandasha](https://avatars.discourse-cdn.com/v4/letter/g/c0e974/32.png) [@gbandasha](https://discuss.elastic.co/u/gbandasha)\
**Post date:** [March 31, 2021, 12:41pm UTC](https://discuss.elastic.co/t/id-value-is-getting-changed-in-elastic/268905/1 "2021-03-31T12:41:26Z")

</div>

I have a unique\_id value in my index and that is also used as the document\_id, but as my document count is increasing drastically I had changed the type from double because that was causing truncation of my values. I changed it to data type "keyword" as many elastic articles suggest that we should only use numeric datatype if we are going to do range searches, aggregation, etc.. which I am not performing on the unique\_id field. But after that, I notice that it is changing the whole value, for example, I send the value "432375692312511746" and in elastic doc, it shows up as "795807590142069243".

I would like to understand why is this happening and also which data type to use for such long numbers and not get into the problem of truncation and change values.

---

<div class="post-metadata">

**Author:** ![gbandasha](https://avatars.discourse-cdn.com/v4/letter/g/c0e974/32.png) [@gbandasha](https://discuss.elastic.co/u/gbandasha)\
**Post date:** [April 5, 2021, 9:47pm UTC](https://discuss.elastic.co/t/id-value-is-getting-changed-in-elastic/268905/2 "2021-04-05T21:47:54Z")

</div>

waiting on the community for response.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [April 5, 2021, 10:40pm UTC](https://discuss.elastic.co/t/id-value-is-getting-changed-in-elastic/268905/3 "2021-04-05T22:40:28Z")

</div>

Hi @gbandasha

Can you please provide the following

The version of elastic you're working with

The mapping for the document

A sample input document

Then what the document looks like after it has been indexed / when you GET it back.

And point out the differences or changes.

---

<div class="post-metadata">

**Author:** ![gbandasha](https://avatars.discourse-cdn.com/v4/letter/g/c0e974/32.png) [@gbandasha](https://discuss.elastic.co/u/gbandasha)\
**Post date:** [April 6, 2021, 11:20pm UTC](https://discuss.elastic.co/t/id-value-is-getting-changed-in-elastic/268905/4 "2021-04-06T23:20:56Z")

</div>

Sure, please find the details below

1. v 7.11.2.

2. I am using the legacy mapping template and the field that I am facing an issue with is unique\_id and the mapping for that is attached below in the sinnpet.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/1/b/1b71c3a7c5454c53586ebfab47a7eb1943e47264.png)

1. Sample JSON data.

{"unique\_id" : "432375692312511746","client\_name" : "test"}

1. In Kibana Discover and even the JSON tab it should up as

{"unique\_id" : " **795807590142069243**","client\_name" : "test"}

Let me know if you need anything else

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [April 6, 2021, 11:49pm UTC](https://discuss.elastic.co/t/id-value-is-getting-changed-in-elastic/268905/5 "2021-04-06T23:49:47Z")

</div>

Hmmmm I just ran this on 7.11.1 and 7.12.0 (I didn't have a 7.11.2 handy)  
Are you sure you are retrieving the same documents?

```
DELETE test

PUT /test
{
  "mappings": {
    "properties": {
      "name" : {"type" : "keyword"},
      "unique_id": {
        "type" : "keyword",
        "eager_global_ordinals": false,
        "norms": false,
        "index": true,
        "store": false,
        "index_options": "docs",
        "split_queries_on_whitespace" : false,
        "doc_values": true
      }
    }
  }
}

POST test/_doc
{
  "name" : "stephen",
  "unique_id" : "432375692312511746"
}

POST test/_doc
{
  "name" : "jeffery",
  "unique_id" : "1293847209184720193847"
}

POST test/_doc
{
  "name" : "dude",
  "unique_id" : "09870987356409586734059867"
}

```

Results they all look correct to me. and In Discover and Kibana

```
GET test/_search

{
  "took" : 2,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 3,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "test",
        "_type" : "_doc",
        "_id" : "scmTqXgBZuwJvuVN3t8C",
        "_score" : 1.0,
        "_source" : {
          "name" : "stephen",
          "unique_id" : "432375692312511746"
        }
      },
      {
        "_index" : "test",
        "_type" : "_doc",
        "_id" : "ssmTqXgBZuwJvuVN3t8m",
        "_score" : 1.0,
        "_source" : {
          "name" : "jeffery",
          "unique_id" : "1293847209184720193847"
        }
      },
      {
        "_index" : "test",
        "_type" : "_doc",
        "_id" : "s8mTqXgBZuwJvuVN3t9J",
        "_score" : 1.0,
        "_source" : {
          "name" : "dude",
          "unique_id" : "09870987356409586734059867"
        }
      }
    ]
  }
}
```

---

<div class="post-metadata">

**Author:** ![gbandasha](https://avatars.discourse-cdn.com/v4/letter/g/c0e974/32.png) [@gbandasha](https://discuss.elastic.co/u/gbandasha)\
**Post date:** [April 7, 2021, 1:16am UTC](https://discuss.elastic.co/t/id-value-is-getting-changed-in-elastic/268905/6 "2021-04-07T01:16:45Z")

</div>

Thanks, @stephenb for the quick response and you are right if I run through DEV tools to add a record or even If I try like 10 - 20 records through logstash it works but when I have like millions of doc's it changes the value.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [April 7, 2021, 1:25am UTC](https://discuss.elastic.co/t/id-value-is-getting-changed-in-elastic/268905/7 "2021-04-07T01:25:45Z")

</div>

Elasticsearch shouldn't be changing values for no reason, that is not to say there might be some strange defect but lets check a few other things.

Perhaps it could be something in your logstash pipeline when it's getting overwhelmed or wrong type setting there.

I am curious about a few of the non default settings, if you just make it a keyword without any of the other settings do you see the same behavior? i.e. just

```auto
DELETE test

PUT /test
{
  "mappings": {
    "properties": {
      "name" : {"type" : "keyword"},
      "unique_id": { "type" : "keyword"}
    }
  }
}

```

Also silly question how do you know you're lining up the input document with what you're seeing in elastic. Is there some other unique ID? in other words how do you know you're comparing to the same documents from source to elasticsearch when you have millions?

---

<div class="post-metadata">

**Author:** ![gbandasha](https://avatars.discourse-cdn.com/v4/letter/g/c0e974/32.png) [@gbandasha](https://discuss.elastic.co/u/gbandasha)\
**Post date:** [April 7, 2021, 6:45pm UTC](https://discuss.elastic.co/t/id-value-is-getting-changed-in-elastic/268905/8 "2021-04-07T18:45:19Z")

</div>

@stephenb I will try the default settings and let you know.

Regarding the second question, I am dealing with transactional data and I have many other elements like account\_number, transaction date time, and the amount that helps me drill down to that specific record to compare.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [April 7, 2021, 7:00pm UTC](https://discuss.elastic.co/t/id-value-is-getting-changed-in-elastic/268905/9 "2021-04-07T19:00:40Z")

</div>

Ok good to know when you inspect the document in kafka is the unique\_id still correct?

Is there any processing between kafka an Elasticsearch?

---

<div class="post-metadata">

**Author:** ![gbandasha](https://avatars.discourse-cdn.com/v4/letter/g/c0e974/32.png) [@gbandasha](https://discuss.elastic.co/u/gbandasha)\
**Post date:** [April 7, 2021, 8:06pm UTC](https://discuss.elastic.co/t/id-value-is-getting-changed-in-elastic/268905/10 "2021-04-07T20:06:53Z")

</div>

In Kafka there is no processing and it is correct I checked the topic data and compared it.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [April 7, 2021, 9:46pm UTC](https://discuss.elastic.co/t/id-value-is-getting-changed-in-elastic/268905/11 "2021-04-07T21:46:30Z")

</div>

What it between Kafka and Elasticsearch and did you try the default mapping?

There is something going on... but if elasticsearch has randomly changing keyword values I think we would be getting a lot of reports on that. Lets keep looking.

---

<div class="post-metadata">

**Author:** ![gbandasha](https://avatars.discourse-cdn.com/v4/letter/g/c0e974/32.png) [@gbandasha](https://discuss.elastic.co/u/gbandasha)\
**Post date:** [April 12, 2021, 5:40pm UTC](https://discuss.elastic.co/t/id-value-is-getting-changed-in-elastic/268905/12 "2021-04-12T17:40:04Z")

</div>

@stephenb I set up the default mapping and resent the data through logstash and now the IDs are not getting changed.

Thanks a lot for your help.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [April 12, 2021, 5:52pm UTC](https://discuss.elastic.co/t/id-value-is-getting-changed-in-elastic/268905/13 "2021-04-12T17:52:45Z")

</div>

Good to know... interesting..., thanks for letting us know it is working.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 10, 2021, 5:53pm UTC](https://discuss.elastic.co/t/id-value-is-getting-changed-in-elastic/268905/14 "2021-05-10T17:53:18Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.