I've already setup a scenario where I get all the sales from a database (input-jdbc) , by polling the database every minute via jdbc schedule parameter.
I send this (output-rabbitmq) to a rabbitmq queue and I subscribe to it and get the sales as soon as they are available.
What I'd like to achieve is a scenario where I only go to the database when I know there's something new.
Imagine I can make my system/database notify when there's new data, and logstash would make the connection.
Inputs generally don't have a way to externally trigger their actions. For inputs with scheduled activity (jdbc, http_poller, etc), we only currently allow cron-like scheduling.
Logstash inputs are capable of terminating and causing Logstash to terminate once all data is processed, but this is uncommon. I'm open to exploring an API for triggering some plugins.
@jordansissel, how do you think this trigger should work ?
What I have in mind is some filter parameter that you could trigger an action to another configured jdbc-input to make the fetch, but don't know how to identify a specific input of a logstash configuration.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.