Hi
I'm using logstash monitoring api's to get to know the performance. I executed this
curl -XGET 'localhost:9600/_node/stats/pipeline?pretty'
{
"host" : "0.0.0.0",
"version" : "5.6.13",
"http_address" : "127.0.0.1:9600",
"id" : "30e9060b-2942-465c-823a-0af6a60549bd",
"name" : "0.0.0.0",
"pipeline" : {
"events" : {
"duration_in_millis" : 180,
"in" : 27,
"out" : 27,
"filtered" : 27,
"queue_push_duration_in_millis" : 23
},
"plugins" : {
"inputs" : [
{
"id" : "207b62600220dddd2e825d5cb829815ae04a2755-1",
"events" : {
"out" : 27,
"queue_push_duration_in_millis" : 23
},
"connections" : 2,
"name" : "syslog",
"messages_received" : 27
} ],
"filters" : [
{
"id" : "207b62600220dddd2e825d5cb829815ae04a2755-6",
"events" : {
"duration_in_millis" : 5,
"in" : 1,
"out" : 1
},
"name" : "ruby"
}
],
"outputs" : [
{
"id" : "207b62600220dddd2e825d5cb829815ae04a2755-33",
"events" : {
"duration_in_millis" : 30,
"in" : 1,
"out" : 1
}
]
},
"reloads" : {
"last_error" : null,
"successes" : 0,
"last_success_timestamp" : null,
"last_failure_timestamp" : null,
"failures" : 0
},
"queue" : {
"type" : "memory"
},
"id" : "main"
}
}
I need below details
- How many events reached logstash from source??
- How much time input plugin took to put it in filter queue?
- How much time filter plugin took to process the event?(cumulative time)
- How much time output plugin took to push the records to elasticsearch??
So I assumed that queue_push_duration_in_millis in input plugin tells how much time it took and duration_in_millis in filter and output plugin, tell me the same time taken stats in each layer
Is that correct?? Whats that events stats on the top(below pipeline)?? Does it have any relation with filter plugin?? Also to get cumulative time taken for all 27 events in filter plugin, Should I sum each filter plugin time taken and get the final value?
