I'm using the jdbc input plugin to feed MySQL data into elasticsearch. I'm using
sql_last_value to send incremental updates to elastic. Is there an automated way to send full updates to elasticsearch say every 24 hours and incremental updates every 5 min?
I know that we can use
clean_run to do that, however the configuration file needs to be changed and is a manual process.
Please let me know your thoughts.
Use two inputs on different schedules?
I have two concerns taking this approach.
I have around 50 indices, so that approach would make the config complex.
What are implications of two output plugins trying to write to elastic at the same time?
I'm thinking if I should set up a cron to delete
.logstash_jdbc_last_run file(s). Has anyone tried that? Any thoughts on this approach?
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.