Re-index API creates multiple tasks

Elastic Stack Elasticsearch
1

Hey all,

ES version: 2.4.3

I have been using re-index API since it's launch in 2.X, it's been working fine until recently/last year. I'm noticing that whenever I execute a reindex request, it starts with one task. After a minute or so reindex API spawns multiple tasks doing the same operation which in-turn puts a lot of load on our server and brings it down.

Request to start reindexing:

POST /_reindex?refresh&wait_for_completition=false
{
	 "conflicts": "proceed",
	 "source": {
			"index": "stage-index-1",
			"size": 10000
	 },
	 "dest": {
			"index": "stage-index-0"
	 }
}

After few minutes status of reindex task:

GET /_tasks/?pretty&detailed=true&actions=*reindex
{
  "nodes": {
    "4_MedlkOWRJ-dXEgd8TYLsg": {
      "name": "stage-search-1b",
      "transport_address": "10.99.111.176:9300",
      "host": "10.91.000.146",
      "ip": "10.91.000.146:9300",
      "attributes": {
        "aws_availability_zone": "us-east-1d",
        "data": "false",
        "zone": "us-east-1d",
        "master": "true"
      },
      "tasks": {
        "4_MedlkOWRJ-dXEgd8TYLsg:16307": {
          "node": "4_MedlkOWRJ-dXEgd8TYLsg",
          "id": 16307,
          "type": "transport",
          "action": "indices:data/write/reindex",
          "status": {
            "total": 1720111,
            "updated": 432,
            "created": 469568,
            "deleted": 0,
            "batches": 48,
            "version_conflicts": 0,
            "noops": 0,
            "retries": 0,
            "throttled_millis": 0,
            "requests_per_second": "unlimited",
            "throttled_until_millis": 0
          },
          "description": "",
          "start_time_in_millis": 1519681855045,
          "running_time_in_nanos": 213645628823
        },
        "5_MeU4OWRJ-dXEgd8TYLsg:17996": {
          "node": "5_MeU4OWRJ-dXEgd8TYLsg",
          "id": 17996,
          "type": "transport",
          "action": "indices:data/write/reindex",
          "status": {
            "total": 1720111,
            "updated": 209021,
            "created": 979,
            "deleted": 0,
            "batches": 22,
            "version_conflicts": 0,
            "noops": 0,
            "retries": 0,
            "throttled_millis": 0,
            "requests_per_second": "unlimited",
            "throttled_until_millis": 0
          },
          "description": "",
          "start_time_in_millis": 1519681975056,
          "running_time_in_nanos": 93635535622
        }
      }
    }
  }
}

This task split from 1-2 tasks gets changed to 4-5 tasks and then host crashes.

I also tried changing scroll size to 5000, but reindex still doesn't finish.

Any help would be greatly appreciated.

Thanks,
Ayush

4 Likes
2

@warkolm or @nik9000 any help here will be appreciated.
Thanks!

3

I have noticed the same behavior and I am running ES 6.2.2

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.