# Logstash metrics don't make sense

**URL:** <https://discuss.elastic.co/t/logstash-metrics-dont-make-sense/48350>\
**Category:** Logstash\
**Created:** [April 25, 2016, 8:40pm UTC](https://discuss.elastic.co/t/logstash-metrics-dont-make-sense/48350 "2016-04-25T20:40:38Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sol\_Lederman](https://avatars.discourse-cdn.com/v4/letter/s/ce73a5/32.png) [@Sol\_Lederman](https://discuss.elastic.co/u/Sol_Lederman)\
**Post date:** [April 25, 2016, 8:40pm UTC](https://discuss.elastic.co/t/logstash-metrics-dont-make-sense/48350/1 "2016-04-25T20:40:38Z")

</div>

I have a 1,000 line sample web log file that I run through logstash and I time the run.

> $ time cat apache\_combined\_log.20160420.1000-lines | ../logstash-2.3.1/bin/logstash -f apache\_logstash.3.conf \> foo

> real 0m25.678s  
> user 0m29.378s  
> sys 0m0.904s

So, it takes almost 26 seconds, including the overhead of logstash starting up, to process 1,000 records. That rate is a bit over 2,000 records per minute, which is really slow.

Here's the output of the metrics filter, showing 1,000 records read and showing rates of 12 to 13 records per minute.

{  
"@version" =\> "1",  
"@timestamp" =\> "2016-04-25T20:29:12.669Z",  
"message" =\> "lxelk1.mydomain",  
"events" =\> {  
"count" =\> 1000,  
"rate\_1m" =\> 11.960577390181204,  
"rate\_5m" =\> 12.785128899681027,  
"rate\_15m" =\> 12.927978024063657  
},  
"tags" =\> [  
[0] "metric"  
]  
}

2,000 records per minute is way different than 13 records per minute.

What am I missing?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [April 27, 2016, 5:53am UTC](https://discuss.elastic.co/t/logstash-metrics-dont-make-sense/48350/2 "2016-04-27T05:53:17Z")

</div>

I'm quite sure the emitted rate is actually per second rather than per minute. It still doesn't quite match though (one-minute rate of ~720 eps), but I suspect that might be because you only ran for about 25 seconds and if you calculate the rate based on sliding one-minute windows I could easily see why 2000 eps would be reported as 720 eps. See the GitHub issue below.

> <https://github.com/logstash-plugins/logstash-filter-metrics/issues/29>

---

<div class="post-metadata">

**Author:** ![Sol\_Lederman](https://avatars.discourse-cdn.com/v4/letter/s/ce73a5/32.png) [@Sol\_Lederman](https://discuss.elastic.co/u/Sol_Lederman)\
**Post date:** [April 27, 2016, 5:02pm UTC](https://discuss.elastic.co/t/logstash-metrics-dont-make-sense/48350/3 "2016-04-27T17:02:15Z")

</div>

Thanks for your reply. I would suggest an addition to the metrics filter document to state that metrics are in records per second.

Last night I loaded 1.2 million records and, looking at those metrics after logstash had been running for a while, the reported one minute rate of around 1,000 is very consistent with my calculation of 1,000 records per second being processed.

So, I'm good. Thanks again for your help.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [April 27, 2016, 8:35pm UTC](https://discuss.elastic.co/t/logstash-metrics-dont-make-sense/48350/4 "2016-04-27T20:35:31Z")

</div>

> I would suggest an addition to the metrics filter document to state that metrics are in records per second.

I've filed a pull request that should clarify the situation.

> <https://github.com/logstash-plugins/logstash-filter-metrics/pull/34>
>
> Several people have been confused about the unit of the rate, so make it clear t…hat it's a per-second rate. Also fix various formatting issues to bring this plugin in line with with others. This addresses parts of the confusion in #29.

---

<div class="post-metadata">

**Author:** ![Sol\_Lederman](https://avatars.discourse-cdn.com/v4/letter/s/ce73a5/32.png) [@Sol\_Lederman](https://discuss.elastic.co/u/Sol_Lederman)\
**Post date:** [April 27, 2016, 9:09pm UTC](https://discuss.elastic.co/t/logstash-metrics-dont-make-sense/48350/5 "2016-04-27T21:09:23Z")

</div>

Thank you.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:00am UTC](https://discuss.elastic.co/t/logstash-metrics-dont-make-sense/48350/6 "2017-07-06T05:00:15Z")

</div>


