Constant timeouts querying Packetbeat indices

We're running one ES/Kibana server and pulling in Filebeat, Metricbeat and Packetbeat data. The Filebeat and Metricbeat indices are a few hundred MB and run fine but Packetbeat indicies are several GB. When we include Packetbeat indices in Discover queries we get timeouts and the CPU spikes for long periods. I can increase the Kibana timeout to mask the problem. I've removed flow data to decrease the size of the indices but even searching for a 30 minute window causes timeouts in Kibana.

Error: Gateway Timeout
at respond (https://es-data.veerotech.net/bundles/vendors.bundle.js?v=16588:111:161556)
at checkRespForFailure (https://es-data.veerotech.net/bundles/vendors.bundle.js?v=16588:111:160796)
at https://servername/bundles/vendors.bundle.js?v=16588:105:285566
at processQueue (https://servername/bundles/vendors.bundle.js?v=16588:58:132456)
at https://servername/bundles/vendors.bundle.js?v=16588:58:133349
at Scope.$digest (https://servername/bundles/vendors.bundle.js?v=16588:58:144239)
at Scope.$apply (https://servername/bundles/vendors.bundle.js?v=16588:58:147018)
at done (https://servername/bundles/vendors.bundle.js?v=16588:58:100026)
at completeRequest (https://servername/bundles/vendors.bundle.js?v=16588:58:104697)
at XMLHttpRequest.xhr.onload (https://servername/bundles/vendors.bundle.js?v=16588:58:105435)

What happens if you run the queries directly against Elasticsearch? What is your Elasticsearch cluster size? I'm wondering if you hit an issue on the Elasticsearch or Kibana side.

We are rookies at this :slight_smile: . Can you point me to where I can find how to do this in the docs please?

So basically the question is how many Elasticsearch nodes have you deployed with how much memory? And what is the size of your indices? https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-stats.html

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.