# How do I bundle logs by request?

**URL:** <https://discuss.elastic.co/t/how-do-i-bundle-logs-by-request/136304>\
**Category:** Kibana\
**Created:** [June 18, 2018, 12:18pm UTC](https://discuss.elastic.co/t/how-do-i-bundle-logs-by-request/136304 "2018-06-18T12:18:09Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Joshua\_Fox](https://avatars.discourse-cdn.com/v4/letter/j/e0b2c6/32.png) [@Joshua\_Fox](https://discuss.elastic.co/u/Joshua_Fox)\
**Post date:** [June 18, 2018, 12:18pm UTC](https://discuss.elastic.co/t/how-do-i-bundle-logs-by-request/136304/1 "2018-06-18T12:18:09Z")

</div>

In Stackdriver Log Viewer, all log-lines within a give HTTP request (in our Google AppEngine app) appear together in a single section. This is very useful in tracking behavior inside a single HTTP request.

How do we do this in Kibana?

 ![separatesections](https://us1.discourse-cdn.com/elastic/original/3X/2/5/25f64cc5dac9ca86b9d0b1c512de5c5f37ef413a.png)

---

<div class="post-metadata">

**Author:** ![yaauie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yaauie/32/23363_2.png) [@yaauie](https://discuss.elastic.co/u/yaauie)\
**Post date:** [June 18, 2018, 10:57pm UTC](https://discuss.elastic.co/t/how-do-i-bundle-logs-by-request/136304/2 "2018-06-18T22:57:20Z")

</div>

Do these log messages have anything in common? e.g., is there a request identifier included as part of the message?

At a previous employer that heavily leveraged micro-services, we included an HTTP Middleware in each of our applications that would find an "activity id" header (or generate a UUID if not present), apply it as metadata to the current request, and ensure any _downstream_ http requests included the header. Our logging layer would also ensure that the activity id for each request was a part of any structured logging messages, which were then consumed by Logstash using a JSON Filter Plugin. This worked really well, since in Kibana, all we had to do was filter down on that single field to get the entire chain of requests.

---

<div class="post-metadata">

**Author:** ![Joshua\_Fox](https://avatars.discourse-cdn.com/v4/letter/j/e0b2c6/32.png) [@Joshua\_Fox](https://discuss.elastic.co/u/Joshua_Fox)\
**Post date:** [June 19, 2018, 6:13am UTC](https://discuss.elastic.co/t/how-do-i-bundle-logs-by-request/136304/3 "2018-06-19T06:13:50Z")

</div>

> is there a request identifier

Yes

> At a previous employer...

Looks good. How did you visually bundle a set of lines?

In other words, it's essential that lines from a single request/activity follow each other sequentially rather than being mixed into other requests. And this sequence of lines should be visually distinguished, perhaps as a foldable section.

---

<div class="post-metadata">

**Author:** ![yaauie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yaauie/32/23363_2.png) [@yaauie](https://discuss.elastic.co/u/yaauie)\
**Post date:** [June 19, 2018, 7:11pm UTC](https://discuss.elastic.co/t/how-do-i-bundle-logs-by-request/136304/4 "2018-06-19T19:11:18Z")

</div>

The visual representation is a bit out of scope for Logstash, which is a tool for data pipelines. I don't personally have an answer in this regard (we used Kibana tooling, but I do not currently know what the state-of-the-art is there).

Once the data is in Elasticsearch, a terms aggregation on the activity id (ordered by minimum timestamp, descending), and within that a top-hits aggregation (ordered by timestamp), could get you the results in the order you mentioned.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 17, 2018, 7:11pm UTC](https://discuss.elastic.co/t/how-do-i-bundle-logs-by-request/136304/5 "2018-07-17T19:11:24Z")

</div>

Closed on July 17, 2018, 7:11pm UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.