How to slow down output ingestion rate with logstash?

Elastic Stack Logstash
1

Hello,

I've millions of records in Elasticsearch and sending outputs as individual files using file output plugin.

I've noticed logstash extraction rate is high and used a sleep plugin to slow down after every few secs or every nth event.

Is there any better way to configure logstash to pull elastic data only after certain duration? Also, if the pipeline is failed for some reason, logstash is executing all the way from the beginning. Is there a way to control with an off-set point so logstash can continue from the breaking point?

2

No. There is an open issue requesting it.

Why do you want to make logstash slower?

1 Like
3

Thanks. I've an custom output plugin where the faster ingestion is having issues with the plugin. So, was looking if there is any configuration to control the logstash ingestion.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.