Our elasticsearch (Ver 7.9) consist of 4 Nodes (2 Hot and 2 Warm ) . We are using auditbeat ,winlogbeat and filebeat for sending data to the nodes. We recently started integrating apache access and error logs using the default Apache module which comes with filebeat. After integrating few servers ( 20 Nos) we started seeing high CPU/Load Average on the node where its creating the primary index. Also the indexing rate for filebeat apache is less compared to other indexes used by Cisco ASA module etc. One thing which i noticed is the number of document is very high compared to other indexes . Below highlighted in red is the filebeat index which has the apache module enabled.
Node 1 (hot) - 8vCPU 32GB Standard SSD 1 TB (M) (Heap 16GB)
Node 2 (hot) - 8vCPU 32GB Standard SSD 1 TB (M) (Heap 16GB)
Node 3 (Warm) - 4vCPU 16GB Throughput Optimized HDD 2 TB (M) (Heap 8GB)
Node 4 (Warm) - 4vCPU 16GB Throughput Optimized HDD 2 TB (Heap 8GB)
All the index will be written to Hot Nodes and then moved to Warm Nodes with ILM policy (50 GB or 30 Days)
From the template setting i can see the number of shards is 1 for filebeat.
Please provide some suggestions to bring down the CPU/Load of the hot nodes