Task Management API

Elastic Stack Elasticsearch
1

Hi
Really looking forward to using the task management API in elastic 2.3, just wondering what jobs are shown in there? The documentation implies that reindex jobs will show up, but I was wondering about other jobs such as:

  • Bulk Inserts
  • Delete by query
  • Update by query
  • Shard (re)allocation
  • Backup jobs
  • Index allocation

Is there a definitive list anywhere?

2

Bulk inserts are not considered a task, but reindex (reindex itself and update by query) and replication are. I guess more things will need to get migrated to the new task infrastructure, like delete by query etc.

3

Sorry, what I said is partly right, and wrong for the rest :slight_smile: let me fix this.

Reindex and update by query are a task, also a cancellable one. In general, each transport action, meaning each api call, becomes automatically a task and can be tracked using the tasks list api. That includes bulk inserts, but also any other api call, like snapshot/restore and delete by query. Delete by query may also be cancellable at some point but it isn't for now.

Replication is also a separate job, which happens in the background and can be tracked.

Let me also remind you that the task infra is experimental, we are working on it actively and many things will still change around it.

4

There's currently no definitive list currently. You could have a look through all the actions and look for their names in the code if you were very ambitious to come up with your own list. For example delete by query is going to be indices:data/write/delete/by_query

Can I ask what you're trying to do with a full list?

5

I'm going to include them in a presentation to the rest of the company on the benefits of elasticsearch 2.3. It'll probably be most of interest to the operations team, to monitor for any unusually long running tasks.

We might build a UI with it, to give us a visualisation of currently executing jobs in elasticsearch. We have something along those lines for SQL server, and it's really useful.

6

Would love to see the possibility to manage task states and recurrent tasks with the task management API. By this, it would be possible to write plugins with ingest actions that can fetch regularly from external sources - like JDBC - formerly known as "river" :slight_smile: