CPU and Load Average high after enabling filebeat Apache module

[ajesh]

Hi There,

Our elasticsearch (Ver 7.9) consist of 4 Nodes (2 Hot and 2 Warm ) . We are using auditbeat ,winlogbeat and filebeat for sending data to the nodes. We recently started integrating apache access and error logs using the default Apache module which comes with filebeat. After integrating few servers ( 20 Nos) we started seeing high CPU/Load Average on the node where its creating the primary index. Also the indexing rate for filebeat apache is less compared to other indexes used by Cisco ASA module etc. One thing which i noticed is the number of document is very high compared to other indexes . Below highlighted in red is the filebeat index which has the apache module enabled.

Node Config

Node 1 (hot) - 8vCPU 32GB Standard SSD 1 TB (M) (Heap 16GB)
Node 2 (hot) - 8vCPU 32GB Standard SSD 1 TB (M) (Heap 16GB)
Node 3 (Warm) - 4vCPU 16GB Throughput Optimized HDD 2 TB (M) (Heap 8GB)
Node 4 (Warm) - 4vCPU 16GB Throughput Optimized HDD 2 TB (Heap 8GB)

All the index will be written to Hot Nodes and then moved to Warm Nodes with ILM policy (50 GB or 30 Days)

From the template setting i can see the number of shards is 1 for filebeat.

Please provide some suggestions to bring down the CPU/Load of the hot nodes

Thanks
Ajesh

[ajesh]

Hello Team,

We are new to Elasticsearch , it started running for only 2 months and we are seeing high CPU on both the hot nodes. Please advise if this config is sufficient for the index rate we are seeing .Do we need to add additional node to handle the CPU Load. Please let me know if you need any more.

Node 1

Node 2