How to find the precise "reindex" by "_tasks" API?

Elastic Stack Elasticsearch
1

many reindex operations may happened at the same time

but the " /_tasks/ " can not distinguish them

may like this
"tasks": {
"N6P1aVclSEibQbEgKSoM4Q:580812": {
"node": "N6P1aVclSEibQbEgKSoM4Q",
"id": 580812,
"type": "transport",
"action": "indices:data/write/reindex",
"status": {
"total": 303600,
"updated": 300,
"created": 0,
"deleted": 0,
"batches": 4,
"version_conflicts": 0,
"noops": 0,
"retries": 0
},
"description": "",
"start_time_in_millis": 1469606609258,
"running_time_in_nanos": 2053612568
},
"N6P1aVclSEibQbEgKSoM4Q:577835": {
"node": "N6P1aVclSEibQbEgKSoM4Q",
"id": 577835,
"type": "transport",
"action": "indices:data/write/reindex",
"status": {
"total": 303600,
"updated": 12000,
"created": 0,
"deleted": 0,
"batches": 121,
"version_conflicts": 0,
"noops": 0,
"retries": 0
},
"description": "",
"start_time_in_millis": 1469606501541,
"running_time_in_nanos": 109770467817
}
}

how can I know what is the reindex doing ?the soure and the dest ?

2

Hi @1222kkk,

as far as I am aware of you cannot find out with the task API which reindexing operation is doing what.

May I ask what's your use case to run a lot of reindex tasks concurrently?

Daniel

3

maybe many old index should be reindex to new ones at the same time ,and I want to monitor the states ,.
so I have to distinguish them from the result of the /_tasks?action=*reindex ,because the result may contain many actions named "indices:data/write/reindex" .so how can i know this reindex action is from "index1"to "index2",and this reindex action represents reindex from "index 3"to "index 4".
my english is really clumsy ,but can you understand it？

4

if i want to monitor the status of the reindex , but I can not distinguish them from the resluts of the "/_task?action=*reindex" ,so the only way is to run only one "reindex" operation at the same time?

5

Hi,

so I have to distinguish them from the result of the
/_tasks?action=*reindex ,because the result may contain many actions
named "indices:data/write/reindex" .so how can i know this reindex
action is from "index1"to "index2"

In the reindex API you can have multiple source indices but only one destination index so you really need to run multiple reindex tasks.

if i want to monitor the status of the reindex , but I can not
distinguish them from the resluts of the "/_task?action=*reindex" ,so
the only way is to run only one "reindex" operation at the same time?

You can do either that or have some client side logic to figure out the task id right after you have submitted a new reindex task. I will ask the colleague that wrote the reindex API later to check whether there is a better way to do that.

Daniel

6

I mean, you can work around this with scripting, especially in the 5.0-alphas because painless is always there and nice. But It probably isn't a good choice because the process isn't really resume-able. Breaking it into chunks helps with that and indexes are natural chunks. The same is true for doing it in parallel.

That'd be running the reindex with ?wait_for_completion=false and capturing the task id. The issue with doing that in the 2.x line is that we don't have task result persistence so once the reindex actually completes it just vanishes. In that latest 5.0-alpha this is fixed by persisting the result to an index.

In 2.x you can use start time/running time to figure out which reindex is which. I admit that isn't great. I'd like to improve the tasks API to return the request that you sent it so you can tell them apart.

1 Like
7

Thank you for your reply. It really inspires me a lot .And now I try to distinguish them by taskID(may be a solution?).
I really hope the improvement will come soon . Thanks for your contribution!