# Calculate time difference between two logs with uniqueid

**URL:** <https://discuss.elastic.co/t/calculate-time-difference-between-two-logs-with-uniqueid/149041>\
**Category:** Logstash\
**Created:** [September 19, 2018, 2:38am UTC](https://discuss.elastic.co/t/calculate-time-difference-between-two-logs-with-uniqueid/149041 "2018-09-19T02:38:33Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![shwesinhan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shwesinhan/32/78663_2.png) [@shwesinhan](https://discuss.elastic.co/u/shwesinhan)\
**Post date:** [September 19, 2018, 2:38am UTC](https://discuss.elastic.co/t/calculate-time-difference-between-two-logs-with-uniqueid/149041/1 "2018-09-19T02:38:33Z")

</div>

hi there

is there anyway to calculate time difference between two logs with uniqueid?

eg i create fields log\_timestamp, log\_type, user\_id, mobile\_no, etc., below logs

`2018-09-19 09:24:57,113 (null) 21 WARN : Test :: vh1o4ctmdpn3aft53o5nx0nh :: User Req : <UserRequest><version>9.7</version><userID>458458000000018</userID><userCode>2C10000299</userCode><desc>123456</desc><desc>123456</desc><mobileNo></mobileNo></UserRequest>`

`2018-09-19 09:24:57,453 (null) 21 WARN : Test :: vh1o4ctmdpn3aft53o5nx0nh :: User Res : <UserResponse><version>9.7</version><userID>458458000000018</userID><userCode>2C10000299</userCode><desc>123456</desc><desc>123456</desc><mobileNo></mobileNo></UserResponse>`

i am in the need to calculate log\_timestamp difference between these two logs using user\_id.  
how to link user\_id for them? my logstash pipeline is eating a lot of logs at the same time.

---

<div class="post-metadata">

**Author:** ![AquaX](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aquax/32/92006_2.png) [@AquaX](https://discuss.elastic.co/u/AquaX)\
**Post date:** [September 19, 2018, 2:09pm UTC](https://discuss.elastic.co/t/calculate-time-difference-between-two-logs-with-uniqueid/149041/2 "2018-09-19T14:09:18Z")

</div>

Use the "elapsed" plugin and set your worker threads to 1 so that multiple threads don't miss the end\_events from the elapsed plugin.  
[Elapsed Plugin - Logstash Docs](https://www.elastic.co/guide/en/logstash/current/plugins-filters-elapsed.html)

---

<div class="post-metadata">

**Author:** ![shwesinhan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shwesinhan/32/78663_2.png) [@shwesinhan](https://discuss.elastic.co/u/shwesinhan)\
**Post date:** [October 5, 2018, 7:34am UTC](https://discuss.elastic.co/t/calculate-time-difference-between-two-logs-with-uniqueid/149041/3 "2018-10-05T07:34:55Z")

</div>

hi @AquaX

thanks very much for advice.  
but i'm being used 2 threads long time ago  
any other options??

---

<div class="post-metadata">

**Author:** ![AquaX](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aquax/32/92006_2.png) [@AquaX](https://discuss.elastic.co/u/AquaX)\
**Post date:** [October 5, 2018, 11:49am UTC](https://discuss.elastic.co/t/calculate-time-difference-between-two-logs-with-uniqueid/149041/4 "2018-10-05T11:49:06Z")

</div>

How long is “long”? You can adjust the timeout setting on the elapsed filter, it will require more events to be held in memory for longer but as long as you don’t restart your instance that should be ok.

I can’t think of any other alternative at the moment unless you do some crazy setup like scanning your log for your starting event, save that to elastic and if you find an ending event save that to a new log file. The new log file will then have its own log stash config that will query elastic for the starting event and then you may be able to do something with it. This is just theory and it would have to be tested... not sure that it will be possible.

---

<div class="post-metadata">

**Author:** ![shwesinhan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shwesinhan/32/78663_2.png) [@shwesinhan](https://discuss.elastic.co/u/shwesinhan)\
**Post date:** [October 9, 2018, 10:39am UTC](https://discuss.elastic.co/t/calculate-time-difference-between-two-logs-with-uniqueid/149041/5 "2018-10-09T10:39:14Z")

</div>

thank you for your adivise @AquaX  
but my logs are consecutive streaming.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 6, 2018, 10:39am UTC](https://discuss.elastic.co/t/calculate-time-difference-between-two-logs-with-uniqueid/149041/6 "2018-11-06T10:39:16Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.