We have set up a Logstash connector to gather data from a DB every day at a fixed time (7 a.m.). Now, we have included the following parameters in the input.conf file (some parameters are masked for obvious security reasons).
What we are trying to achieve is the following: the connector should try and contact the DB at 7 a.m as specified in the schedule parameter). If the connection fails, it should then try 5 more connection attempts (as specified in the connection_retry_attempts parameter). Between each attempt, there should be a waiting time of 300 seconds (ad specified in the connection_retry_attempts_wait_time parameter). If at any point the connection is accepted and the data is ingested, the connector should not attempt to connect to the DB anymore. The whole process should than start again the following day at 7 a.m.
What we are getting instead is the following: the connector tries indeed to connect to the DB at 7.am. However, it will just try to do one single connection attempt, regardless if the connection is established or not. So it doesn't seem to be taking the connection_retry_attempts and connection_retry_attempts_wait_time parameters into account. In fact, I'm pretty sure that we could comment out those parameters and the behavior of the connector would not change anyway.
Now, we thought in the beginning that the connection_retry_attempts_wait_time was too short (it was set to 30 seconds, which is actually less time than the query takes to run). However, after changing it to 300 seconds (as shown in the picture above) nothing seems to have changed. I have found a couple of discussions about this on the web and even a GitHub issue has been opened in the past (Retry Establishing Connection if it Fails · Issue #91 · logstash-plugins/logstash-input-jdbc · GitHub). However, no one seems to have found a solution for this (or I have not been able to find it).
So, is it possible to achieve the behavior we want by changing the connector's configuration? And if it is, how should we modify our input.conf file so that it works as expected?
Thanks for your help,