Retry reindexing starting from a certain point

Elastic Stack Elasticsearch
1

Hello. It sometimes happen that reindexing fails, for one reason or another (mostly elastic thread pool's task queue full). I'm working on improving performance in some ways, to run into the issue less often; but I don't think I will ever be able to safely assume that it works every time. So I would like to be able to restart from where it failed when there is an issue. Is that possible?

More in detail, the failure message looks something like:

{
  "took": 292958,
  "timed_out": false,
  "total": 13920455,
  "updated": 0,
  "created": 1147000,
  "deleted": 0,
  "batches": 1147,
  "version_conflicts": 0,
  "noops": 0,
  "retries": {
    "bulk": 0,
    "search": 0
  },
  "throttled_millis": 0,
  "requests_per_second": -1,
  "throttled_until_millis": 0,
  "failures": [
    {
      "shard": -1,
      "reason": {
        "type": "es_rejected_execution_exception",
        "reason": "rejected execution of org.elasticsearch.transport.TransportService$7@680ae7d on EsThreadPoolExecutor[search, queue capacity = 1000, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@3bd943c0[Running, pool size = 19, active threads = 19, queued tasks = 391, completed tasks = 2297158742]]"
      }
    }
  ]
}

Those lines:

  "total": 13920455,
  "updated": 0,
  "created": 1147000,
  "deleted": 0,
  "batches": 1147,

tell me where it stopped. Is it possible to restart the process and specify something like "start at batch 1148"?

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.