# Mean Time Between Documents

**URL:** <https://discuss.elastic.co/t/mean-time-between-documents/136564>\
**Category:** Elasticsearch\
**Created:** [June 19, 2018, 8:11pm UTC](https://discuss.elastic.co/t/mean-time-between-documents/136564 "2018-06-19T20:11:46Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![jduquette](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jduquette/32/26243_2.png) [@jduquette](https://discuss.elastic.co/u/jduquette)\
**Post date:** [June 19, 2018, 8:11pm UTC](https://discuss.elastic.co/t/mean-time-between-documents/136564/1 "2018-06-19T20:11:47Z")

</div>

I am trying to determine the meantime between records, per month, for a set of documents. All documents have a timestamp, and I want to capture the average time between the timestamps on a monthly basis. Have tried several trial and error approaches, but nothing is giving me exactly what I want.

Doc 1:  
{  
"\_index": "my\_index",  
"\_type": "CO",  
"\_source": {  
"timestamp": "2018-01-02T14:00:00",  
"segment": "none",  
"category": "Normal",  
"plannedStart": "2018-01-02T14:00:00",  
"subCategory": "Normal"  
}  
}

Doc 2:  
{  
"\_index": "my\_index",  
"\_type": "CO",  
"\_source": {  
"timestamp": "2018-01-05T14:00:00",  
"segment": "none",  
"category": "Normal",  
"plannedStart": "2018-01-05T14:00:00",  
"subCategory": "Normal"  
}  
}

serial\_diff seems to get me the closest to getting the deltas between the docs, but how to take the next step to aggregate the average of the deltas?  
"aggs": {  
"2": {  
"date\_histogram": {  
"field": "timestamp",  
"interval": "day",  
"time\_zone": "America/New\_York",  
"min\_doc\_count": 1  
},  
"aggs": {  
"start": {  
"sum":{  
"field": "timestamp"  
}  
},  
"delta": {  
"serial\_diff":{  
"buckets\_path":"start",  
"lag": 1  
}  
}  
}  
}  
}

---

<div class="post-metadata">

**Author:** ![Igor\_Motov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_motov/32/45193_2.png) [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)\
**Post date:** [June 19, 2018, 8:59pm UTC](https://discuss.elastic.co/t/mean-time-between-documents/136564/2 "2018-06-19T20:59:06Z")

</div>

Could you define what you mean by "mean time between documents"? The way I understand it if you get documents that arrive in moments`t[0], t[1], t[2].... t[N-1], t[N]` you want to find `((t[1]-t[0])+(t[2]-t[1])+ ... + (t[N]-t[N-1]))/N`?

---

<div class="post-metadata">

**Author:** ![jduquette](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jduquette/32/26243_2.png) [@jduquette](https://discuss.elastic.co/u/jduquette)\
**Post date:** [June 19, 2018, 10:53pm UTC](https://discuss.elastic.co/t/mean-time-between-documents/136564/3 "2018-06-19T22:53:59Z")

</div>

That is exactly right. Sum of all deltas over total document count: Σ[(t[N] - t[N-1])]/N

---

<div class="post-metadata">

**Author:** ![Igor\_Motov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_motov/32/45193_2.png) [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)\
**Post date:** [June 19, 2018, 11:26pm UTC](https://discuss.elastic.co/t/mean-time-between-documents/136564/4 "2018-06-19T23:26:48Z")

</div>

In this case isn't it the same thing as

```auto
(t[1]-t[0]+t[2]-t[1]+ ... + t[N]-t[N-1])/N = (t[N] - t[0])/N

```

Which, assuming a pretty frequent arrival of events within time period `T`, basically assuming that the first event arrived pretty close to the beginning of period and the last event arrived pretty close to the end of the period, we can say that `(t[N] - t[0]) ≈ T`, especially if `T` and `N` are pretty large. Then the average time between events will be just `T / N`. You know `T` - it's 1 month in your case and you can get `N` by just running date histogram on your documents. Am I missing something?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 17, 2018, 11:40pm UTC](https://discuss.elastic.co/t/mean-time-between-documents/136564/5 "2018-07-17T23:40:59Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.