How to monitor logstash itself?


(Michael Scofield) #1

Are there any tools that can monitor logstash's health? I googled it but found no answers.


(Magnus Bäck) #2

The following discussion from a day or two ago might be helpful. If not, please be more specific in what you're after.


(Michael Scofield) #3

Thanks! That discussion is enlightening. However, I am looking for some metrics like "how many events are received/sent per second" or "how many bytes are received/sent per second".


(Magnus Bäck) #4

Well, if you run statsd you can have Logstash ping it for each message and you'll at least get events/s.


(Michael Scofield) #5

Thanks for the reply, but I'm not very clear about how to "ping" the statsd.

Do you mean that I have to include the statsd output plugin and have it feed with some event field like timestamp?


(Magnus Bäck) #6

Do you mean that I have to include the statsd output plugin and have it feed with some event field like timestamp?

Not the timestamp since that's implied but you could send metrics to a counter-type metric named logstash.events or whatever you want, and stastd will periodically emit both the absolute count and the rate.


(Michael Scofield) #7

Ah, thanks! :slight_smile:


(Mark Walkom) #8

LS 2.0 will have this sort of thing exposed :slight_smile:


(Michael Scofield) #9

Thx! Really looking forward to 2.0, it's better to have these kinds of metrics got from logstash itself.

Is there any plan releasing 2.0?


(Mark Walkom) #10

I'm not sure of the timeline on that one sorry.


(Andrew Cholakian) #11

@MichaelScofield we're still working on figuring out the timeline at the moment. There will def be an announcement when we have one however!


(M1k3ga) #12

According to the statsd doc it uses the %{host} field as logging host.
In our setup with many logstash shippers and indexers the host is always the name of the original host where the logfile was read from.
How do you distinguish between each instance for monitoring each logstash instance?

We use the metrics plugin with environment plugin ($HOSTNAME) to trigger a heartbeat event from each logstash instance every 60 seconds.

But statsd seems to be interesting, i will have a look at it.


(system) #13