# How to Aggregate Log Content **URL:** \ **Category:** Logstash\ **Created:** [July 17, 2015, 9:27pm UTC](https://discuss.elastic.co/t/how-to-aggregate-log-content/25833 "2015-07-17T21:27:09Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![Dilip2610](https://avatars.discourse-cdn.com/v4/letter/d/aeb1de/32.png) [@Dilip2610](https://discuss.elastic.co/u/Dilip2610)\ **Post date:** [July 17, 2015, 9:27pm UTC](https://discuss.elastic.co/t/how-to-aggregate-log-content/25833/1 "2015-07-17T21:27:09Z")
Hi, I have few log entries as below. Thu Jul 09 2015 15:00:39 [serviceout][0x8060015e][crypto][info] mpgw(mpgwService): tid(6440055)[request][255.255.215.129] gtid(6440055): Data encryption succeeded Thu Jul 09 2015 15:00:39 [serviceout][0x8060010b][crypto][info] mpgw(mpgwService): tid(6440055)[request][255.255.215.129] gtid(6440055): certificate validation succeeded Thu Jul 09 2015 15:00:39 [serviceout][0x80e00073][latency][info] mpgw(mpgwService): tid(6440055)[255.255.215.129] gtid(6440055): Latency: 0 3 0 2 3 2 0 398 399 398 399 399 399 398 2 3 [[https://HOSTNAME](https://HOSTNAME):PORT/URI] How can i aggregate three entries using logstash Filter based on gtid and pick the URL from last entry. what i am trying is as follows. - Do Grok Parse the message and create field that has gtid. - Do Grep to pick corresponding entries based on tid assign it to a field ---\> **looking suggestion to implement this.** Thanks, Dilip ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [July 18, 2015, 2:51am UTC](https://discuss.elastic.co/t/how-to-aggregate-log-content/25833/2 "2015-07-18T02:51:37Z")
Maybe a multiline filter? But that might be a bit clunky ☹ I'll wait for Magnus to drop in and provide some advice 😉 ---
**Author:** ![Dilip2610](https://avatars.discourse-cdn.com/v4/letter/d/aeb1de/32.png) [@Dilip2610](https://discuss.elastic.co/u/Dilip2610)\ **Post date:** [July 20, 2015, 2:58pm UTC](https://discuss.elastic.co/t/how-to-aggregate-log-content/25833/3 "2015-07-20T14:58:01Z")
Hi, Thanks for your suggestion. I tried with Multi-line, but its not working as expected. Is there any other way to achieve Thanks, Dilip ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [July 20, 2015, 3:11pm UTC](https://discuss.elastic.co/t/how-to-aggregate-log-content/25833/4 "2015-07-20T15:11:03Z")
Since the last line (with the URL) is basically a superset of the preceding lines, what kind of aggregation are you looking to do? That is, given the three-line snippet in your post, what is the expected outcome? ---
**Author:** ![Dilip2610](https://avatars.discourse-cdn.com/v4/letter/d/aeb1de/32.png) [@Dilip2610](https://discuss.elastic.co/u/Dilip2610)\ **Post date:** [July 20, 2015, 3:41pm UTC](https://discuss.elastic.co/t/how-to-aggregate-log-content/25833/5 "2015-07-20T15:41:46Z")
> [@Dilip2610](#): > > what i am trying is as follows. > > Do Grok Parse the message and create field that has gtid. > Do Grep to pick corresponding entries based on tid assign it to a field ---\> looking suggestion to implement this. Hi Magnus, My Implementation is as follows. what i am trying is as follows. Do Grok Parse the message and create field that has gtid. Do Grep to pick corresponding entries based on gtid Pick the URL from Latency record corresponding to gtid. Create URL field for all three log entries. so when the 3 log entries are shipped to Elastic Search, i should have URL filed for all the three log entries. For Latency record it will not be an issue, but for other two log records i need to create URL field with the URL value from Latency record. Thanks, Dilip ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [July 20, 2015, 4:25pm UTC](https://discuss.elastic.co/t/how-to-aggregate-log-content/25833/6 "2015-07-20T16:25:55Z")
Oh, okay. There's no stock Logstash plugin that can help you here. I'd either write a custom plugin or have the third message sent to a broker and write a small service that consumes those messages and updates the remaining two entries in ES with the URL. ---
**Author:** ![Dilip2610](https://avatars.discourse-cdn.com/v4/letter/d/aeb1de/32.png) [@Dilip2610](https://discuss.elastic.co/u/Dilip2610)\ **Post date:** [July 20, 2015, 4:39pm UTC](https://discuss.elastic.co/t/how-to-aggregate-log-content/25833/7 "2015-07-20T16:39:43Z")
Plugin might be helpful in this case. Might be useful for others as well Just out of curiosity can logstash-filter-aggregate suits for this scenario. Thanks, Dilip ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [July 20, 2015, 4:51pm UTC](https://discuss.elastic.co/t/how-to-aggregate-log-content/25833/8 "2015-07-20T16:51:46Z")
I don't think the aggregate filter helps here. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 5:34am UTC](https://discuss.elastic.co/t/how-to-aggregate-log-content/25833/9 "2017-07-06T05:34:12Z")