Your Query Is Taking While-Kibana/Report download (Aggregation on documents with unique id more than 20000)

[PRASHANT_MEHTA]

Hello All,

Issue 1:

In my dashboard I've intrgrated 5 visuals,4 bar charts and 1 enhanced data table.
While I select time range for 2 week,it fetches around 18000 unique documents that is visible in enhanced data table and above 4 bar charts are also made with info displayed in table.

The issue I'm facing is that it give me error:Your query is taking time.I would like to understand how could i resolve this and what might be the cause.
Single elastic node cluster with 8 core cpu and 8gb jvm memeory allocated as its test instance.

Table logic:
Fetching top hit values of four fileds and bucketing on unique id[bucket size edited to 25000]
Would like to fetch more no of documents but want to avoid latency in response,like in discover data comes in seconds.

Issue 2: Cant generate execl reports for this becoz of size limit of 10000 by deafult.Best approach to resolve this.Would like to download whole data of 25000 records one clicked on download button in excel(button angular logic).[backend java api used,unable to bypass more than 10000 records]

Four similar bar charts based on table and filtered to show relevant data according to usecase

@fbaligand Firstly thanx for this plugin as its quite useful for advanced usecases.
Could you please suggest something on this?,Is there size limitation for no of documents fetched or something could be done to fetch large amount of aggregated data with less latenc,sometime page is unresponsive.

Kindly guide.

Thanks

[fbaligand (Fabien Baligand)]

Hi,
If you want to get one line per document, I strongly recommend to use "Document Table" visualization. That is another visualization provided by "enhanced-table" plugin (with "Enhanced Table" visualization).

"Enhanced Table" viz is done to show aggregated results.
"Document Table" viz s done to show non-aggregated results (1 row = 1 document).

Doing so, you can get 25 000 documents and more, because plugin will do a simple search query, and not an aggregation query.
And if you check 'Full CSV export' in "Options" tab, you can then download all documents from Elasticsearch to CSV file, whatever document count.

Finally, to get all features and best performance, be sure to use the latest plugin version (1.13.3 for now).