About Increasing Logstash Performance


(oscar) #1

Hi,

I have a logstash agent that extracts data from redis, preprocesses it and reengines it in redis again.

I have been able to incorporate in this agent the metric filter to give me an idea of ​​how fast the process is.

Approximately, I have an average yield of 4000 processed trails per second.

The logstash version I work with in 5.1.1 and the redis version 3.2.5.

The installation I made of logstash is intended to be deployed several agents of the same type within an agent directory, so that the same instance of logstash raise N agents, as many as I declare within the agents directory.

The problem I have is that I do not see that it is increasing at all the speed of process of logstash increasing the number of agents of logstash in the instance that I raise.

The same is an error in my perception of how the performance increase works.

As part of improving the logstash agent itself to make it faster, if I raise more "equal" agents in the same instance, do I expect a global increase in processing?

What other alternatives would you have?


(Guy Boertje) #2

Redis is single threaded. Each client request to Redis will be handled serially.
The logstash redis input can read in larger batches. It uses a LUA script to read a batch. In Redis only one LUA script can run at a time.
The redis output writes in batches too.

You need to understand the how much time it takes for the Logstash(es) feeding data to Redis take to build up their batch and how long it takes for the redis output to write and the same for the Logstash(es) reading data from Redis.
For best performance a good interleave of writes and reads must be seen.

                  ______________~~~~~~~~~~~~~~~~~~~~~~~____~~~~~~~~~~~~~______~~~~~~~~~~~~~~~~~~~~~~~____________
Logstash writer 1 | build batch |write|  |    build batch  |write|  |    build batch    |write|
Logstash writer 2       | build batch |write|  |    build batch  |write|  |    build batch    |write|
Logstash reader 1                           |read| process batch + ES out |  |read| process batch + ES out|
Logstash reader 2                                |read| process batch + ES out|    |read| process batch + ES out|

In the above ascii diagram, the read and write durations are small - if build batch or process batch + ES out durations are much longer than read or write then you should get few waits in Redis.

Obviously the more LS instances you have (redis clients really) with simple filter and output sections the more waits at Redis you will have.

I think if you use Kafka then you have to worry less about parallel reads and writes.


(oscar) #3

Thx so much for this information guyboertje. :wink:


(Guy Boertje) #4

You are welcome. More info in this blog post https://www.elastic.co/blog/just_enough_redis_for_logstash


(system) closed #5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.