Cpu usage is high

Elastic Stack Elasticsearch
1

Hi,
We are using AWS managed Elasticsearch of instance type r4.xlarge with 4 nodes available in 2 zones and EBS storage size per node is 1000GB.
Version of Elasticsearch is 6.2.
Data streaming into ES is done via logstash and it is realtime streaming data.
We are holding the logs on day based indices and only 30 days data will be available.Each day data holds a max of 45gb and we are using terms, date_histogram, cardinality aggregation queries(on 30 indices).For every second, we are making 400 requests, which has the combination of above mentioned aggregations on the 30 indices data.We are facing the high CPU utilisation often.

Please help us in reducing the cpu utilisation

2

400 requests sounds like a quite high request volume for a cluster that size, especially as it seems you have considerably more data than fits on the operating system page cache. This could result in a lot of disk I/O, but as you are using AWS Elasticsearch service it may be hard to get insight into this. Apart from optimizing your queries and/or mappings I can not think of any easy quick fixes. If the cluster can not handle the load I would recommend either switching to instances with faster storage, e.g. i3 instances, or scale up or out the cluster.