Sample Dashboard Display Issues

Recently setup an ELK Stack and am having issues after I began feeding it MetricBeat and WinlogBeat data from five servers. When pulling up MetricBeat's sample Host or Overview dashboards I receive one of two types of errors for the modules.

rejected execution of org.elasticsearch.common.util.concurrent.TimedRunnable@5f708265 on QueueResizingEsThreadPoolExecutor[search, queue capacity = 1000, min queue capacity = 1000, max queue capacity = 3000, frame size = 2000, targeted response rate = 1s, task execution EWMA = 1.3ms, adjustment amount = 50, QueueResizingEsThreadPoolExecutor[search, queue capacity = 1000, org.elasticsearch.common.util.concurrent.QueueResizingEsThreadPoolExecutor@10fa6c61[Running, pool size = 7, active threads = 7, queued tasks = 1322, completed tasks = 118090]]]

or

The aggregations key is missing from the response, check your permissions for this request.

Prior to including these five servers, these dashboards loaded fine with just data from the ELK Stack server itself. I've adjusted the max queue capacity from 1000 to 3000 but it hasn't made much of an impact. Below are the server specs, what can I do to resolve this?

VMWare VM
Server 2012 R2
12GB RAM
4 vCPUs
ElasticSearch heap set to 4GB
LogStash head set to 2GB
Also increased elasticsearch.requestTimeout to 60s in Kibana.yml

Well, I made a few ElasticSearch configuration modifications, not sure if this is recommended or not but it has made a significant difference. I read the recommended size multiplier was 5 for each core but 25 allowed slightly faster queries, 30 tanked performance.

thread_pool:
search:
size: 25
queue_size: 6000
min_queue_size: 4000
max_queue_size: 6000
target_response_time: 1s

I also modified Kibana's timeout setting to 60s which helped a lot but there was still the occasional timeout. Increasing it to 120s has resolved timeouts, although that is a LONG time to wait.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.