# Durations and counts between events

**URL:** <https://discuss.elastic.co/t/durations-and-counts-between-events/182324>\
**Category:** Kibana\
**Created:** [May 22, 2019, 9:10pm UTC](https://discuss.elastic.co/t/durations-and-counts-between-events/182324 "2019-05-22T21:10:20Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![cheeni](https://avatars.discourse-cdn.com/v4/letter/c/7c8e57/32.png) [@cheeni](https://discuss.elastic.co/u/cheeni)\
**Post date:** [May 22, 2019, 9:10pm UTC](https://discuss.elastic.co/t/durations-and-counts-between-events/182324/1 "2019-05-22T21:10:20Z")

</div>

Greetings Elastic Stack experts.

I have an index of timestamped events like so (which I have transformed into JSON, but using simple strings here for readability):

ID: 1, name: ABC, time: 111.11, action: start  
ID: 2, name: DEF, time: 112.0, action: start  
ID: 1, name: , time: 114, action: join  
ID: 1, name: ABC, time: 123, action: end  
  
ID: 1, name: ABC, time: 249, action: start  
ID: 7, name: XYZ, time: 250.0, action: start  
ID: 1, name: , time: 260, action: end

I need to compute elapsed times and join counts for each occurrence of ID: 1, and so on. A summary along these lines would be even better:

ID: 1, name: ABC, start\_times: [111.11, 249], end\_times: [123, 260], join\_times: [114]

Is it possible to do this in Kibana, maybe using a script?

The logstash elapsed filter did not give me I did try the elapsed filter in Logstash but it resulted in some errors due to which I had to manually validate a whole bunch of data.

Any guidance?

Thanks

---

<div class="post-metadata">

**Author:** ![jpcarey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jpcarey/32/46668_2.png) [@jpcarey](https://discuss.elastic.co/u/jpcarey)\
**Post date:** [May 22, 2019, 10:46pm UTC](https://discuss.elastic.co/t/durations-and-counts-between-events/182324/2 "2019-05-22T22:46:55Z")

</div>

You would likely need to use a [scripted metric aggregation](https://www.elastic.co/guide/en/elasticsearch/reference/7.1/search-aggregations-metrics-scripted-metric-aggregation.html), and then use the Vega visualization to run that scripted aggregation and display how you would like. I will warn, you are essentially going to be building the aggregation and the visualization - aka, this will not be easy.

---

<div class="post-metadata">

**Author:** ![cheeni](https://avatars.discourse-cdn.com/v4/letter/c/7c8e57/32.png) [@cheeni](https://discuss.elastic.co/u/cheeni)\
**Post date:** [May 23, 2019, 2:48am UTC](https://discuss.elastic.co/t/durations-and-counts-between-events/182324/3 "2019-05-23T02:48:23Z")

</div>

Thanks Jared. Can you suggest an alternate? Maybe filters in Logstash?

---

<div class="post-metadata">

**Author:** ![jpcarey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jpcarey/32/46668_2.png) [@jpcarey](https://discuss.elastic.co/u/jpcarey)\
**Post date:** [May 23, 2019, 4:20pm UTC](https://discuss.elastic.co/t/durations-and-counts-between-events/182324/4 "2019-05-23T16:20:21Z")

</div>

The `logstash-filter-aggregate` provides some good examples that fit in with your scenario. It does come with some limitations though:

> You should be very careful to set Logstash filter workers to 1 ( `-w 1` flag) for this filter to work correctly otherwise events may be processed out of sequence and unexpected results will occur.

> **[Aggregate filter plugin | Logstash Reference \[8.11\] | Elastic](https://www.elastic.co/guide/en/logstash/current/plugins-filters-aggregate.html)**

If you know that this type of data will come from a single source, you can isolate it to a dedicated logstash instance (or a logstash pipeline with the pipeline.workers set to 1. Theoretically should work, but I've never actually tried with the aggregate filter)

---

<div class="post-metadata">

**Author:** ![cheeni](https://avatars.discourse-cdn.com/v4/letter/c/7c8e57/32.png) [@cheeni](https://discuss.elastic.co/u/cheeni)\
**Post date:** [May 23, 2019, 6:26pm UTC](https://discuss.elastic.co/t/durations-and-counts-between-events/182324/5 "2019-05-23T18:26:39Z")

</div>

Thanks Jared. I'll try it out.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 20, 2019, 6:26pm UTC](https://discuss.elastic.co/t/durations-and-counts-between-events/182324/6 "2019-06-20T18:26:43Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.