# Calculating duration between two documents in Elasticsearch

**URL:** <https://discuss.elastic.co/t/calculating-duration-between-two-documents-in-elasticsearch/92858>\
**Category:** Elasticsearch\
**Created:** [July 12, 2017, 6:19pm UTC](https://discuss.elastic.co/t/calculating-duration-between-two-documents-in-elasticsearch/92858 "2017-07-12T18:19:33Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![tarp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tarp/32/51884_2.png) [@tarp](https://discuss.elastic.co/u/tarp)\
**Post date:** [July 12, 2017, 6:19pm UTC](https://discuss.elastic.co/t/calculating-duration-between-two-documents-in-elasticsearch/92858/1 "2017-07-12T18:19:33Z")

</div>

Hi,

I'm collecting logs from an application recording the start time. Then another record is signifying the end time. Both records have to match on DeviceId and a CorrelationId. I'm using an ingest pipeline to process the logs. When the log line is processed that is the end time, I want to query ES and get the starttime and calculate the difference, and add a field for it.  
I'm at a loss on how and if this can be done in pipelines in Elasticsearch.  
I saw something similar via Logstash, but I'm trying to avoid Logstash and just do this in Elasticsearch.

Any ideas?  
Thanks,  
Tim

---

<div class="post-metadata">

**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\
**Post date:** [July 12, 2017, 8:53pm UTC](https://discuss.elastic.co/t/calculating-duration-between-two-documents-in-elasticsearch/92858/2 "2017-07-12T20:53:15Z")

</div>

Use an entity centric index - see [https://youtu.be/yBf7oeJKH2Y](https://youtu.be/yBf7oeJKH2Y)

The comments include a link to example data and scripts

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 9, 2017, 8:53pm UTC](https://discuss.elastic.co/t/calculating-duration-between-two-documents-in-elasticsearch/92858/3 "2017-08-09T20:53:41Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

---

<div class="post-metadata">

**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\
**Post date:** [June 20, 2018, 10:22am UTC](https://discuss.elastic.co/t/calculating-duration-between-two-documents-in-elasticsearch/92858/4 "2018-06-20T10:22:13Z")

</div>

> [@Mark\_Harwood](#):
>
> The comments include a link to example data and scripts

No they don't because I expect they are awaiting a review which is why only I can see them.

I've just updated the scripts to work with elasticsearch 6.3 and [this is the link](http://bit.ly/entcent6_3)