Just wanted to see what people are using out of interest to schedule and manage orchestration of logstash configurations?
Yes, I know logstash has a schedule future with cron like syntax but wanted to manage the flow of different pipelines.
Today I use S3 to store all my configuration files and last run files and have period syncs. From a scheduling and orchestration layer we use run-deck. This is ok - but looking at options how I address the transient nature of EC2 (e.g. what if my EC2 dies).
Interested to know how people manage configurations & last run files as well.
I am thinking right now of experimenting with EFS and Airflow.
All the best