How can I delete all the duplicate records except one to keep my data?

Elastic Stack Elasticsearch
9

Thanks David and Veera

I've created this script with reindex and processor but i still don't have my hash in _id. Do you know what can be wrong in my script ?

#!/bin/bash

curl -X DELETE http://10.10.10.21:9200/blockchain3?pretty && \
curl -XPUT -H "Content-Type: application/json" http://10.10.10.21:9200/blockchain3 -d @/apps/code/elasticsearch/blockchain/mapping.json

curl -XPUT -H "Content-Type: application/json" http://10.10.10.21:9200/_ingest/pipeline/set_id?pretty -d '
{
  "description": "sets the value of _id from the field hash",
  "processors": [
    {
      "set": {
        "field": "_id",
        "value": "{{hash}}"
      }
    }
  ]
}'

curl -XPOST -H "Content-Type: application/json" http://10.10.10.21:9200/_reindex?pretty -d '
{
  "source": {
    "index": "blockchain"
  },
  "dest": {
    "index": "blockchain3"
  }
}'