Delete_by_query number of documents Error

Elastic Stack Elasticsearch
1

Hi, I run this code:

POST topf/_delete_by_query?conflicts=proceed&max_docs=500000
{
  "query": {
    "range": {
      "time": {
        "gte": "2020-09-18T03:00:00",
        "lte": "2020-09-18T04:00:00"
      }
    }
  }
}

I get this response:

{
  "took": 197,
  "timed_out": false,
  "total": 500000,
  "deleted": 0,
  "batches": 1,
  "version_conflicts": 0,
  "noops": 0,
  "retries": {
    "bulk": 0,
    "search": 0
  },
  "throttled_millis": 0,
  "requests_per_second": -1,
  "throttled_until_millis": 0,
  "failures": [
    {
      "index": "topf",
      "type": "_doc",
      "id": "5P9Jn3QB8h4zPTyXs00T",
      "cause": {
        "type": "illegal_argument_exception",
        "reason": "number of documents in the index cannot exceed 2147483519"
      },
      "status": 400
    },
.
.
.
]
}

my index setting:
docs num 2097483519
primary shard: 1
replica shard: 0

How can i save my index and delete old docs?
i dont want delete whole of my index.
any solutions?

2

You should reindex in new indices using the reindex API and then drop the old index

3

Hi @dadoonet,
Are you sure? Is there any other solutions?
I have 80GB free space, and index volume is 255GB. So i must buy new HDD and increase my HDD free space :frowning:

4

I feel you have allowed the single index to grow huge and can't do much operations in that. (May be ILM or curator to rollover when you create it, so it rolls over correctly), so it is not held as a single index but more like topf_2020-12-01.00000 etc.
I can't see any other options other than dadoonet mentioned

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.