Hi,
I know that with a cron-like syntax we can schedule a jdbc input to run regularly, however our underlying database tables (postgres) are refreshing irregularly, therefore we are looking for a way to only run the SQL query defined in the jdbc input in an ad-hoc manner.
Ideally we wouldn't want to run Logstash from the command line with -f flag for the config config file, as it spins up an individual instance from logstash with it's JVM and since we have some bigger table we've set the heap size to be larger than the default, but at the same time we have like 200 database tables and some might get refreshed at the same time. The best would be to have a single pipeline continuously running with upstart, and within that pipeline trigger the input whenever we want instead of a static cron schedule.
Any ideas how to implement such, or any alternative way to achieve what we want?
Many thanks