I kept seeing mentions of a monitoring API when LS 2.0 hit, but cant seem to find any mention of it in the current docs. Did it get implemented?
Bummer. Looks like it got pushed out to LS 3.0? I'm guessing there's not even an ETA for that version.
Is there a way to see how many lines are queued up and waiting to move out of LS? I've been using rsyslog and rely on its ability to provide queue stats but I'd like to move to something a bit more powerful.
Queued up where? Logstash has no internal queue to speak of.
If Logstash cant write to its destination, it would in theory be queued from the filter to output, right? I'm basing this knowledge off of the roadmap verbiage here: https://www.elastic.co/guide/en/logstash-roadmap/current/index.html#_resiliency
Basically, I want to know if logstash is behind on sending a fast filling file to its destination for whatever reason (usually because the destination cant handle the ingestion rate).
If Logstash cant write to its destination, it would in theory be queued from the filter to output, right?
Yes and no. There are two 20-event buffers in the pipeline but that's all there is in terms of queueing. In practice a clogged output will stall the whole pipeline.
Basically, I want to know if logstash is behind on sending a fast filling file to its destination for whatever reason (usually because the destination cant handle the ingestion rate).
Logstash won't give you such metrics, but since file inputs are observable from the outside you could compare the sincedb offset to the file size and see how far behind Logstash is. See Logstash parsing progress bar - Stack Overflow.
Thanks for that stack overflow link. I might be able to cobble something together using that.
Do you know if these are the kinds of metrics that are planned with the monitoring API?
Do you know if these are the kinds of metrics that are planned with the monitoring API?
I don't know. Have you checked the GitHub issues?
I don't have a date for 3.X, but it won't be far off.