Usage of logstash metrics to compute the number of my logs/events


(shruthi santosh) #1

Hi,

We are using the generator plugin to get the events.rate_1m/logs.rate_1m metrics and trying to send it to statsd through the statsd plugin. The rate that we see is too high as show in the screenshot below and gives the rate of logs/events that is generated by the generator,

We have used the examples provided in the documentation to get to this logstash config.The config that we are using is as follows,
input {
generator {
type => "generated"
}
}
filter {
if [type] == "generated" {
metrics {
meter => "events"
add_tag => "metric"
}
}
}
output {
if "metric" in [tags] {
stdout {
codec => line {
format => "rate: %{events.rate_1m}"
}
}
statsd {
host => "127.0.0.1"
port => 8125
gauge => [ "events.rate_1m", "%{events.rate_1m}" ]
sender => "eventbroker"
}
}
}

As an end user of Logstash I do not want to know how the generator input plugin is implemented or how many times the generator is called. I want to count what matters to me, that is, the number of MY log/events that arrive at logstash input. So I think that the generator input plugin is not as intuitive as it should be. this is a bug fix / feature enhancement that should be looked at.

Thanks
Shruthi


(system) #2