Dear Logstash Team,
I've grown to love Elastic Stack and we're now adopting and seeing the power of it. I'd love to see the following features:
- For Record Last Run & Meta Data Path. This is a great way of tracking your last process point to pick up again on your next run/schedule.
In our automated build pipeline, I have Spinnaker deploying a fedora AMI with logstash installed and configured as a systemd process. It does its thing logstash starts, health endpoints resolve and my conf processes (all great). But, what if i have since processed 1 million messages and the instance terminates. By design of logstash it will check for the file, if not exist create and set default value to either 0 or 1-1-1970 (depending on the datatype of that field)
The ask: It would be nice to have an ability to direct this to either DB (RDS / NoSQL) or S3 or alternative mechanism that isn't local to the instance
- Everything is log file and console driven.
I have a logstash AMI running separately in AWS which is pushing events to Hosted ES. These are all text based configuration files.
It would be nice e.g. Kibana to have the ability to see your logstash configurations, last runs and metrics that can be used for debugging purposes. With the improvements adding in 5.x i see a world where your APIs ship those metrics to provide a nice monitoring UI.
Long term, i'd love to see the ability to handle this all UI manner with functionality such as managed ES logstash instances. This would be another revenue source for you and reduce the need client side to spin up the infrastructure etc.
thanks and welcome to opinions on my asks.