Async-search keepalive value

Elastic Stack Elasticsearch
1

Hi,

As far as I can see from Async search | Elasticsearch Reference [7.11] | Elastic - the default keep-alive value for async search is 5 days and is regulated on the query side.

This results in .async_search index on our CCS nodes growing to unreasonable size.
Is it possible to change the default value of the keep_alive parameter for async searches, so the .async_search index does not grow in size that much?
I would say that 1 minute is more than enough for all our use-cases.

Best regards,
Alex

2

I suggest submit an issue in elasticsearch github repo requesting to add a node level setting, something like async_search.default_keep_alive that can be dynamically configurable.

3

Mayya,
Thank you for the recommendation.
As a workaround now I would consider creating an ILM policy to delete .async-search index every day.
Could you please let me know whether there are some ramifications of this approach except the obvious ones (async searches around the time of the rotation might not yield full results or can fail)

4

You should not touch any system indices including .async_search.
In the future all direct access to system indices will be prohibited.
Currently, you can still manually delete it, but it will be recreated a next time async search is run.

5

Mayya,

In that case - what should we do with the .async-search index?
It grows up to 70+ Gb currently, filling up the disks on our cross-cluster search nodes - and resulting in inability to perform searches.
We do not know how big this index will grow - neither we have any form of control over this growth.
Also - it is created with only 1 shard - so it can overgrow Elastic official recommendations in regards to the shard size (How to size your shards | Elasticsearch Reference [7.11] | Elastic)
As far as I can see - we will have to create a template for managing this system index anyways.
Maybe we can use ILM with rollover to control the sharding, index size and the duration of the storage?
Could you please let me know if there is any ETA for the system indices access restrictions?

6

Could you please let me know if there is any ETA for the system indices access restrictions?

I don't have any update on the ETA.

Where are your async searches are coming from? From Kibana? Or you are manually running them?

7

As far as I can see - most of the async searches are coming from Kibana (although our users can also run manual async searches via Elasticsearch API)
Currently we have 11203 documents (and 284 deleted ones) in .async-search index, which results in 13.2gb primary size.

8

Unfortunately I don't know if Kibana allows to disable async searches or set up a custom keep_alive parameter. Might work asking in the Kibana section of discuss.

There is a way to delete async search (including its results stored in .async-search index), but you need to know specific ids of these async searches.

9

Thank you for the recommendations.
Am I understanding correctly that there is no easy way to list all IDs of stored searches in the index using async_search API?

10

there is no easy way to list all IDs of stored searches in the index using async_search API

No, there is no way to list them.

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.