How can I actually track the start time, Stop Time, records transferred, and maybe errors on a single pipeline run? I do not see a stopped time. I do know if I stop logstash and restart the service that it writes to the logs "started" but that isnt the actual start time of the ingest.
I've looked out using outputs but again nothing seems to be working as expected for some reason.
Any thoughts would be greatly appreciated.
The monitoring API allows you to see how many events an input has generated. Depending on how often you sample the data that might allow you to infer start/stop times and event counts. You can use metricbeat to collect that data and Kibana to visualise it.
Ah you want LS monitoring. As Badger said, URL will provide LS pipeline monitoring. Check this how to set the monitoring.
Also you can track the monitoring in Kibana->Stack Monitoring->Logstash. It will be easier to track if you set plugin's IDs