# Calculating delay of sending/processing delay of events

**URL:** <https://discuss.elastic.co/t/calculating-delay-of-sending-processing-delay-of-events/48799>\
**Category:** Beats\
**Created:** [April 29, 2016, 11:42am UTC](https://discuss.elastic.co/t/calculating-delay-of-sending-processing-delay-of-events/48799 "2016-04-29T11:42:32Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![hartfordfive](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hartfordfive/32/44794_2.png) [@hartfordfive](https://discuss.elastic.co/u/hartfordfive)\
**Post date:** [April 29, 2016, 11:42am UTC](https://discuss.elastic.co/t/calculating-delay-of-sending-processing-delay-of-events/48799/1 "2016-04-29T11:42:32Z")

</div>

We currently have an ELK cluster which supports multiple tenants and each tenant sends their logs to our Logstash cluster via the Filebeat shipper. We try to provide our tenants with stats of what the average processing delay is for their pipeline (each tenant has their own lumberjack input) although this approach isn't completely accurate, and requires additional setup.

We would like a way to provide tenants with more accurate stats that could be built in to each event being shipped by Filbeat. For example, say there was an option "enable\_event\_stats: true" which could be specified in the prospector YAML config, each event could have a "\_filebeat\_sent\_time" field added to it with a timestamp corresponding to the time just prior to it being sent. Once the data reaches Logstash, we could simply calculate the delay between the current time and the "\_filebeat\_sent\_time" timestamp on the event.

Is this a feature that could be added? It would simplify things in our case and I'm sure others would take advantage of this if it were available.

Thanks

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [April 30, 2016, 4:14pm UTC](https://discuss.elastic.co/t/calculating-delay-of-sending-processing-delay-of-events/48799/2 "2016-04-30T16:14:37Z")

</div>

Doesn't Filebeat add a `@timestamp` field when it reads the file? If so you can subtract that timestamp from the current timestamp on the Logstash side to get the delay. You probably need a ruby filter for this.

---

<div class="post-metadata">

**Author:** ![hartfordfive](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hartfordfive/32/44794_2.png) [@hartfordfive](https://discuss.elastic.co/u/hartfordfive)\
**Post date:** [May 2, 2016, 12:09pm UTC](https://discuss.elastic.co/t/calculating-delay-of-sending-processing-delay-of-events/48799/3 "2016-05-02T12:09:09Z")

</div>

Thank you @magnusbaeck, I didn't realize that filebeat added a '@timestamp' field to events the prospector picks up. From what I can tell from looking in the Logstash source code, [this line](https://github.com/elastic/logstash/blob/065a82467a1ca43d2ba52e9a0ef0108b6bde0c52/logstash-core-event/lib/logstash/event.rb#L75) indicates Logstash only adds it's a '@timestamp' field if that value isn't already set?

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [May 2, 2016, 4:53pm UTC](https://discuss.elastic.co/t/calculating-delay-of-sending-processing-delay-of-events/48799/4 "2016-05-02T16:53:19Z")

</div>

right, filebeat (and all other beats) do send events with `@timestamp` being set. It's the time the events was originally acquired (time filebeat did read line from file). With `@timestamp` already set, it will not be overwritten by logstash.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 9:52pm UTC](https://discuss.elastic.co/t/calculating-delay-of-sending-processing-delay-of-events/48799/5 "2017-07-05T21:52:36Z")

</div>


