# Organising multiline events with multithreaded log lines

**URL:** https://discuss.elastic.co/t/organising-multiline-events-with-multithreaded-log-lines/103040
**Category:** Logstash
**Created:** [October 6, 2017, 3:43pm UTC](https://discuss.elastic.co/t/organising-multiline-events-with-multithreaded-log-lines/103040 "2017-10-06T15:43:01Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![A.McBride](https://avatars.discourse-cdn.com/v4/letter/a/f17d59/32.png) [@A.McBride](https://discuss.elastic.co/u/A.McBride)
#### Post date: [October 6, 2017, 3:43pm UTC](https://discuss.elastic.co/t/organising-multiline-events-with-multithreaded-log-lines/103040/1 "2017-10-06T15:43:02Z")

</div>

I've been trying to combine log lines based on a thread number, but the problem is that multiple processes write to a single log file, meaning we have process blocks interlaced with other process blocks. I want to be able to combine them into multiline events based on thread number, parse them and ship them to elasticsearch  
Here's a simplified example of our log lines:

2017-08-19 14:00:01,851 DEBUG thread-19 Method begin  
2017-08-19 14:00:01,852 DEBUG thread-19 Log lines  
2017-08-19 14:00:01,853 DEBUG thread-19 Log lines  
2017-08-19 14:00:01,854 DEBUG thread-20Method begin  
2017-08-19 14:00:01,855 DEBUG thread-20 Log lines  
2017-08-19 14:00:01,856 DEBUG thread-19 Method End  
2017-08-19 14:00:01,857 DEBUG thread-20 Method End

Ideally I'd like to have them grouped like this:

2017-08-19 14:00:01,851 DEBUG thread-19 Method begin  
2017-08-19 14:00:01,852 DEBUG thread-19 Log lines  
2017-08-19 14:00:01,853 DEBUG thread-19 Log lines  
2017-08-19 14:00:01,856 DEBUG thread-19 Method End  
2017-08-19 14:00:01,854 DEBUG thread-20Method begin  
2017-08-19 14:00:01,855 DEBUG thread-20 Log lines  
2017-08-19 14:00:01,857 DEBUG thread-20 Method End

However I'm not sure how to go about doing this. I thought stream\_identity would be a good fit but the multiline filter was deprecated some time ago. Is there a way to do this or is it a pointless endeavor? Let me know if there's anything I haven't explained properly.  
Many thanks!

---

<div class="post-metadata">

### Author: ![jakelandis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jakelandis/32/36163_2.png) [@jakelandis](https://discuss.elastic.co/u/jakelandis)
#### Post date: [October 6, 2017, 4:23pm UTC](https://discuss.elastic.co/t/organising-multiline-events-with-multithreaded-log-lines/103040/2 "2017-10-06T16:23:52Z")

</div>

You may want to take a look at [https://www.elastic.co/guide/en/logstash/current/plugins-filters-aggregate.html](https://www.elastic.co/guide/en/logstash/current/plugins-filters-aggregate.html)

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 3, 2017, 4:23pm UTC](https://discuss.elastic.co/t/organising-multiline-events-with-multithreaded-log-lines/103040/3 "2017-11-03T16:23:59Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
