Search Behavior/Performance

Elastic Stack Kibana
1

Not sure if this belongs in the Kibana or Elasticsearch section...

I currently have 33 hosts reporting Winlogbeat and MetricBeat data to a single node ElasticStack setup. Total index size for these two comes in at about 30GB per day and about 100 million documents. Our VM has lots of fast storage, 20GB of RAM and 6 2.4GHz CPUs. If I try to look at 24 hours worth of metrics using the [Metricbeat System] Overview example dashboard, I get timeouts on the search (Kibana is configured with a two minute timeout). Same behavior with the WinlogBeat example dashboard.

Three questions come from this:

  1. When I specify a time frame, is Kibana only looking at a part of the index that spans that time, the whole index, or all indexs that match the index pattern? (In my case, metricbeat-* or winlogbeat-*)

  2. Is this kind of performance in line with expectations?

  3. Other than building up/out the infrastructure, is there anything I can do to improve search performance?

2

Which version are you on? When you are querying and seeing timeouts, what does CPU usage, disk I/O and iowait look like?

3

Running 6.1.1 on Server 2012 R2. According to the perfmon metrics in the pictures below, it appears pretty obvious where the bottleneck is. How can I improve performance or are people deploying 12 CPU ElasticStack boxes? Would it be better to have two or three 2 CPU boxes than one 6 CPU box?

4

I do not see any pictures, so can not tell what the bottleneck is.

5

I guess it would be helpful to attach them...

Baseline
Baseline.JPG1236×628 94.8 KB

24HrSearch
24HrSearch.JPG1271×523 37.3 KB

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.