How do I increase the number of logs per second processing by Elastic Agent and Ingest pipeline?

I came first and only received the Suricata log from the elastic agent, about 600 EPS processing is done. The amount of CPU that you can see in the following figure by htop command:

Then I came and processed other logs along with the Suricata log by the elastic agent (such as zeek, syslog , etc.). In this case, the processing EPS of the Suricata log dropped sharply (it became about 100 EPS) while the CPU usage rate remained the same:

This indicates that the number of pipeline threads is probably constant or that the elastic agent has a limited processing power.

How can I increase processing throughput? Are there any settings for ingest pipeline or elastic agent? Or am I facing another problem?

I had created another issue in this regard: What determines the maximum output speed (log per second) of the elastic agent?


Instead of using the elastic agent, I came and used the logstash and set the output to the pipeline inside the elastic, in this case the processing reaches up to 4000 events per second, so the problem is not the elastic pipeline but the elastic agent.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.