Reindex stops stops after 1000 docs

Elastic Stack Elasticsearch
1

Hello!

I currently have 1 index filled with about 500k documents that I want to reindex into another index where I have added a semantic text search field. It is the most basic kind that uses the built in stuff in Elasticsearch. I then start a reindex task like so:

POST _reindex?wait_for_completion=false
{
  "source": {
    "index": "old_index",
    "size": 1000
  },
  "dest": {
    "index": "new_index"
  }
}

but the indexing task stops at 1000 and I do not understand why. If I cancel the task and then try to start a new one it refuses to do anything. I've looked in these places and it looks fine:

GET _nodes/stats/fs

GET _cluster/settings

GET _cat/shards?v=true

GET _nodes/stats/thread_pool?filter_path=**.write

GET _cluster/allocation/explain

When I began doing this we where on the standard plan but now we have upgraded to the Platinum plan and I even get notices that it want me to try out adding semantic text indices.

Any ideas of what the issue could be or where I could go look dig deeper?

2

Hello @honungsburk ,

Welcome !

What is the status of the task? Can you show us the information of the task return?

3

Running "GET _tasks/WzNWSxXjRu-9Zx2jTZ-WPQ:88103689" I get this

{
  "completed": false,
  "task": {
    "node": "WzNWSxXjRu-9Zx2jTZ-WPQ",
    "id": 88103689,
    "type": "transport",
    "action": "indices:data/write/reindex",
    "status": {
      "total": 483903,
      "updated": 0,
      "created": 1000,
      "deleted": 0,
      "batches": 2,
      "version_conflicts": 0,
      "noops": 0,
      "retries": {
        "bulk": 0,
        "search": 0
      },
      "throttled_millis": 0,
      "requests_per_second": -1,
      "throttled_until_millis": 0
    },
    "description": "reindex from [frank_index_se] to [frank_semantic_index_se]",
    "start_time_in_millis": 1740036138574,
    "running_time_in_nanos": 21751935165,
    "cancellable": true,
    "cancelled": false,
    "headers": {
      "trace.id": "7c46730bdfe52fc12d58f4cd933d7356"
    }
  }

It never completes batch 2. I've tried deleting the semantic index and recreating it but it does not have any impact either

4

I found the problem in the logs: there was no ML node in the cluster.

1 Like