_id value is getting changed in elastic

Elastic Stack Elasticsearch
7

Elasticsearch shouldn't be changing values for no reason, that is not to say there might be some strange defect but lets check a few other things.

Perhaps it could be something in your logstash pipeline when it's getting overwhelmed or wrong type setting there.

I am curious about a few of the non default settings, if you just make it a keyword without any of the other settings do you see the same behavior? i.e. just

DELETE test

PUT /test
{
  "mappings": {
    "properties": {
      "name" : {"type" : "keyword"},
      "unique_id": { "type" : "keyword"}
    }
  }
}

Also silly question how do you know you're lining up the input document with what you're seeing in elastic. Is there some other unique ID? in other words how do you know you're comparing to the same documents from source to elasticsearch when you have millions?