# Measuring the taking time

**URL:** <https://discuss.elastic.co/t/measuring-the-taking-time/113900>\
**Category:** Logstash\
**Created:** [January 3, 2018, 10:43am UTC](https://discuss.elastic.co/t/measuring-the-taking-time/113900 "2018-01-03T10:43:18Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![shwesinhan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shwesinhan/32/78663_2.png) [@shwesinhan](https://discuss.elastic.co/u/shwesinhan)\
**Post date:** [January 3, 2018, 10:43am UTC](https://discuss.elastic.co/t/measuring-the-taking-time/113900/1 "2018-01-03T10:43:18Z")

</div>

hi everyone!

is there any way to do measuring the time taken by logstash pipeline processing (input cloudwatch logs \> filter some patterns \> output to aws elasticsearch)????  
i am using logstash v6.1.1 and not include x-pack in kibana.

plz, point out to me  
thank you

---

<div class="post-metadata">

**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\
**Post date:** [January 3, 2018, 12:31pm UTC](https://discuss.elastic.co/t/measuring-the-taking-time/113900/2 "2018-01-03T12:31:46Z")

</div>

There is an API you can use CURL or similar with. [https://www.elastic.co/guide/en/logstash/current/node-stats-api.html#pipeline-stats](https://www.elastic.co/guide/en/logstash/current/node-stats-api.html#pipeline-stats)

---

<div class="post-metadata">

**Author:** ![shwesinhan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shwesinhan/32/78663_2.png) [@shwesinhan](https://discuss.elastic.co/u/shwesinhan)\
**Post date:** [January 4, 2018, 2:25am UTC](https://discuss.elastic.co/t/measuring-the-taking-time/113900/3 "2018-01-04T02:25:37Z")

</div>

thanks @guyboertje

1. 

could you explain about "duration\_in\_millis" and "queue\_push\_duration\_in\_millis"

which one is for input/filter/output duration time?

1. i am not clear duration\_in\_millis is added with previous value or new value?  
eg:

> {  
> "host" : "UAT-LOGSTASH-01",  
> "version" : "6.1.1",  
> "http\_address" : "127.0.0.1:9600",  
> "id" : "4f8524dc-38ac-4f3e-9563-bf965573761d",  
> "name" : "UAT-LOGSTASH-01",  
> "events" : {  
> "in" : 2000,  
> "filtered" : 2000,  
> "out" : 2000,  
> "duration\_in\_millis" : 4284,  
> "queue\_push\_duration\_in\_millis" : 620  
> }  
> }

> {  
> "host" : "UAT-LOGSTASH-01",  
> "version" : "6.1.1",  
> "http\_address" : "127.0.0.1:9600",  
> "id" : "4f8524dc-38ac-4f3e-9563-bf965573761d",  
> "name" : "UAT-LOGSTASH-01",  
> "events" : {  
> "in" : 5000,  
> "filtered" : 5000,  
> "out" : 5000,  
> "duration\_in\_millis" : 9493,  
> "queue\_push\_duration\_in\_millis" : 824  
> }  
> }

previous value ==\> "duration\_in\_millis" : 4284,  
next value ===\> "duration\_in\_millis" : 9493, (added with 4234 or new value?)

appreciate!!

---

<div class="post-metadata">

**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\
**Post date:** [January 4, 2018, 8:57am UTC](https://discuss.elastic.co/t/measuring-the-taking-time/113900/4 "2018-01-04T08:57:07Z")

</div>

Sure, I will try.

My understanding is that the `duration_in_millis` is the total time taken to process `events in|filtered|out`.

5000 / 9493 = 0.527 kEps or 527 events per second.

`queue_push_duration_in_millis` is the accumulative time the input are waiting to push events into the queue.

824 / 5000 = 0.185 ms or 185 microseconds on average. A high value here indicates a form of back-pressure where the outputs are slower than the inputs. Your value seems low, indicating that the outputs are keeping pace with the input cadence.

---

<div class="post-metadata">

**Author:** ![shwesinhan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shwesinhan/32/78663_2.png) [@shwesinhan](https://discuss.elastic.co/u/shwesinhan)\
**Post date:** [January 4, 2018, 9:15am UTC](https://discuss.elastic.co/t/measuring-the-taking-time/113900/5 "2018-01-04T09:15:25Z")

</div>

understand.  
thank you @guyboertje

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 1, 2018, 9:15am UTC](https://discuss.elastic.co/t/measuring-the-taking-time/113900/6 "2018-02-01T09:15:44Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.