Dear LogstashTeam,
I hope this message finds you well. I am currently using Logstash version 8.4.3 with OpenSearch and have come across a behavior in the OpenSearch output plugin that I believe could be improved for better error handling and system stability.
According to the documentation and source code, in scenarios where network reachability to OpenSearch is lost, the plugin attempts retries indefinitely. While I understand the intention behind ensuring data is not lost due to temporary network issues, this behavior presents a challenge during prolonged network outages or server issues. In such cases, the indefinite retries can lead to unnecessary resource consumption and potential system instability.
Would it be possible to introduce a configuration parameter that allows users to set a maximum number of retry attempts? This feature would provide better control over the plugin's behavior in error scenarios, allowing for a more predictable and manageable system behavior, especially in production environments.
Introducing such a parameter could allow users to balance between ensuring data delivery and managing system resources effectively, without manual intervention during extended periods of network unavailability.
I appreciate your consideration of this request and look forward to any guidance or updates you could provide on this matter.
Best regards,
nw-engineer
)