How can I actually track the start time, Stop Time, records transferred, and maybe errors on a single pipeline run? I do not see a stopped time. I do know if I stop logstash and restart the service that it writes to the logs "started" but that isnt the actual start time of the ingest.
I've looked out using outputs but again nothing seems to be working as expected for some reason.
Any thoughts would be greatly appreciated.
This is the only log that I see coming out that makes sense. The others are started and shut down. Which I believe are both related to starting / stopping the logstash service
The monitoring API allows you to see how many events an input has generated. Depending on how often you sample the data that might allow you to infer start/stop times and event counts. You can use metricbeat to collect that data and Kibana to visualise it.
Ah you want LS monitoring. As Badger said, URL will provide LS pipeline monitoring. Check this how to set the monitoring.
Also you can track the monitoring in Kibana->Stack Monitoring->Logstash. It will be easier to track if you set plugin's IDs
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.