# Help in capturing E2E timestamp from raw logs

**URL:** <https://discuss.elastic.co/t/help-in-capturing-e2e-timestamp-from-raw-logs/351335>\
**Category:** Logstash\
**Created:** [January 18, 2024, 9:24am UTC](https://discuss.elastic.co/t/help-in-capturing-e2e-timestamp-from-raw-logs/351335 "2024-01-18T09:24:25Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Anurag101](https://avatars.discourse-cdn.com/v4/letter/a/6f9a4e/32.png) [@Anurag101](https://discuss.elastic.co/u/Anurag101)\
**Post date:** [January 18, 2024, 9:24am UTC](https://discuss.elastic.co/t/help-in-capturing-e2e-timestamp-from-raw-logs/351335/1 "2024-01-18T09:24:25Z")

</div>

Hi All,

I am new to ELK and am trying to achieve a real time monitoring framework which captures E2E timestamp of an ansync logback application.  
The ask is to capture the timestamp corresponding to a unique ID in the logs for start and end of the processing of the correlation ID logged in the a distributed log file system.  
I need some help in finding right direction to getting this implemented

---

<div class="post-metadata">

**Author:** ![strawgate](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/strawgate/32/131008_2.png) [@strawgate](https://discuss.elastic.co/u/strawgate)\
**Post date:** [January 23, 2024, 12:21am UTC](https://discuss.elastic.co/t/help-in-capturing-e2e-timestamp-from-raw-logs/351335/2 "2024-01-23T00:21:52Z")

</div>

It sounds like this usecase would benefit from the aggregate plugin [Aggregate filter plugin | Logstash Reference [8.12] | Elastic](https://www.elastic.co/guide/en/logstash/current/plugins-filters-aggregate.html)

Specifically example 1 in the document sounds similar to what you're trying to do.

---

<div class="post-metadata">

**Author:** ![Anurag101](https://avatars.discourse-cdn.com/v4/letter/a/6f9a4e/32.png) [@Anurag101](https://discuss.elastic.co/u/Anurag101)\
**Post date:** [January 23, 2024, 7:39am UTC](https://discuss.elastic.co/t/help-in-capturing-e2e-timestamp-from-raw-logs/351335/3 "2024-01-23T07:39:05Z")

</div>

@strawgate Thank you for your response, the example pathway looks promising I will try implementing that within the test ecosystem I have.  
The only doubt/confusion I have is each ID is a unique value which is randomly generated within the application, how will I be able to compare these random values and then aggregate the timestamps.

Also one more question would be in linux I was searching within the logs using regexp and was selecting a specific group which yielded me the relevant data, how can i achieve this in elastic.

---

<div class="post-metadata">

**Author:** ![strawgate](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/strawgate/32/131008_2.png) [@strawgate](https://discuss.elastic.co/u/strawgate)\
**Post date:** [January 23, 2024, 1:29pm UTC](https://discuss.elastic.co/t/help-in-capturing-e2e-timestamp-from-raw-logs/351335/4 "2024-01-23T13:29:44Z")

</div>

Hi, you can use a grok filter to parse the lines using regex.

With aggregate you can use any field parsed from the log, ID is just an example but you can use IP address or anything else

---

<div class="post-metadata">

**Author:** ![Anurag101](https://avatars.discourse-cdn.com/v4/letter/a/6f9a4e/32.png) [@Anurag101](https://discuss.elastic.co/u/Anurag101)\
**Post date:** [January 23, 2024, 5:41pm UTC](https://discuss.elastic.co/t/help-in-capturing-e2e-timestamp-from-raw-logs/351335/5 "2024-01-23T17:41:18Z")

</div>

Thank you so much for the responses, I am in the process of implementation as advised 🙂

---

<div class="post-metadata">

**Author:** ![strawgate](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/strawgate/32/131008_2.png) [@strawgate](https://discuss.elastic.co/u/strawgate)\
**Post date:** [January 23, 2024, 5:41pm UTC](https://discuss.elastic.co/t/help-in-capturing-e2e-timestamp-from-raw-logs/351335/6 "2024-01-23T17:41:55Z")

</div>

Happy to help!

Feel free to share your progress and any issues you run into along the way!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 20, 2024, 5:42pm UTC](https://discuss.elastic.co/t/help-in-capturing-e2e-timestamp-from-raw-logs/351335/7 "2024-02-20T17:42:52Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
