Can i using 'Scroll API' to elasticsearch with vega in kibana?

[teemoGod (teemo park)]

hi. im using elasticsearch / kibana 6.4.1

im using built-in vega in kibana.

im using deep sub-aggregation query. (almost 7 depth)

but there are some performance issue. too slow to use multiple aggregation.

so, i thinking about 'how about only use 'hits' field'.

but, 'query' query's max size is 10000.

but i need more datas.

can i using scroll api in built-in vega in kibana?

[tsullivan (Tim Sullivan)]

This doesn't seem possible as with the Scroll API, you need to capture a scroll cursor and supply it to multiple searches to get a chunk of hits at a time, and then clear the scroll when it's done. Vega would have to be able to "detect" that you're trying to use the scroll API and do it's own cursor management and re-query for you. It doesn't do that: it will give ES what you tell it to, but it doesn't know your intention.

Also take note, from Request body search | Elasticsearch Guide [8.11] | Elastic

Scrolling is not intended for real time user requests, but rather for processing large amounts of data, e.g. in order to reindex the contents of one index into a new index with a different configuration.

It sounds like this advice is for you: if the amount of data you have is too large to query in the form that it is, it could be re-processed and have the raw stats rolled up into summarized forms that can actually be visualized.

[teemoGod (teemo park)]

@tsullivan

thx for your response and thx for your advice.

so helpful and hopeful idea.