The internal Logstash batch size does not need to match what is used for the inputs, so having it set to a smaller value should not be a problem. I therefore do not understand why it needs to be 4000. I also find it hard to understand exactly what you have done so am not sure I will be able to assist unless you provide a much more detailed description.
When it comes to the batches of 50.100, have you measured that with the monitoring API? With which settings are that? With the paging enabled? When I run my pipeline for 6 minutes the JDBC input doesn't give me more than 4k results p/s
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.