Message "Your query is taking a while"

Elastic Stack Kibana
1

Hi people, I've implemented the netflow module with Filebeat. In my ELK 7.8.0 I'm receiving a lots of flows from a BGP router, and when I go to the Filebeat Netflow dashboards and I query for a long date range (24 hs and more), the query is too slow and I get this message:

"Your query is taking awhile"

No data and geolocalization are displayed.

And after that I see this error:

RequestTimeoutError: Request timed out
at https://siem.company.com/31997/bundles/plugin/data/data.plugin.js:6:422367
at e._subscribe (https://siem.company.com/31997/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:271:897125)
at e._trySubscribe (https://siem.company.com/31997/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:21:56087)
at e.subscribe (https://siem.company.com/31997/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:21:55873)
at e.call (https://siem.company.com/31997/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:271:225206)
at e.subscribe (https://siem.company.com/31997/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:21:55744)
at https://siem.company.com/31997/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:21:56745
at new Promise ()
at e.toPromise (https://siem.company.com/31997/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:21:56710)
at SearchSource._callee2$ (https://siem.company.com/31997/bundles/plugin/data/data.plugin.js:6:94803)
If the query is for 1 hour, 6 hours, 10 hours, it works OK and I can retrieve all the flow data and their geolocalization.

I have 10 CPU's, 20 GB RAM, and -Xms10g m / -Xmx10g for java heapspace.

I send all netflow data to the same filebeat index as the other incoming events. from remote machines with Filebeat installed.

Can you help me please ?

Special thanks !

2

What is the size of you indices and shards? What type of storage do you have? What is the indexing rate?

3

Dear Christian, thanks for your response.

Please can you tell me how I should calculate the size of my indices and shards, and the indexing rate?

My storage is not a SSD disk yet, because it's a test scenario.

If I put all netflow events in the same index of the filebeat events from remote machines, can this be a problem ?

Because when I put all network events in a separate index called "netflow-xxxx", the geolocalization doesn't work because it wait for filebeat data matching.

Thanks again, cheers !

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.