# Parsing multiline log

**URL:** https://discuss.elastic.co/t/parsing-multiline-log/149539
**Category:** Logstash
**Created:** [September 22, 2018, 10:18am UTC](https://discuss.elastic.co/t/parsing-multiline-log/149539 "2018-09-22T10:18:25Z")
**Posts on this page:** 2
**Page:** 1

<div class="post-metadata">

### Author: ![tito32](https://avatars.discourse-cdn.com/v4/letter/t/d26b3c/32.png) [@tito32](https://discuss.elastic.co/u/tito32)
#### Post date: [September 22, 2018, 10:18am UTC](https://discuss.elastic.co/t/parsing-multiline-log/149539/1 "2018-09-22T10:18:25Z")

</div>

HI!  
I have some huge multiline logs from vendor programm with different info on each line. All lines have several patterns to match. Almost each line starts with INFO\DEBUG word after that it has some datetime numbers pid of process and than finally some important or meaningless info. In one line there is some id's in others there are timestamp of starting or ending event. So I want to parse it that way so i can find in elastic by id of events duration of some processes that are shown in log and so on. Is there a way to I can do that with logstash?

I've tried elapsed plugin already. I can get some timing by using it but can't add id's because it's in different lines.

Now i trying multiline codec but it seems to be not that i want because it just store all lines in one field and if i use something like  
"if grokeparserfailure" and than add tag  
so I'll have very much to compute.

I have aroung 100G logs\day.

Logs example:

DEBUG 2018/08/14 12:30:03.389 58830 Start event\*  
...  
INFO 2018/08/14 12:30:03.389 58830 Receive new message\*  
INFO 2018/08/14 12:30:03.389 58830 Message found\*  
DEBUG 2018/08/14 12:30:03.393 58830 About to Process my message\*  
INFO 2018/08/14 12:30:03.393 58830 _keyword1_Start process time that i need to calc\*  
...  
DEBUG 2018/08/14 12:30:03.395 58830 Many meaningless lines  
...  
INFO 2018/08/14 12:30:03.398 58830 _keyword2_Exit process\*[%Some uniq id here%]  
...  
DEBUG 2018/08/14 12:30:03.399 58830 400 more lines  
...  
INFO 2018/08/14 12:30:03.507 58830 Work Commited\*  
INFO 2018/08/14 12:30:03.507 58830 Stop event\*  
...  
DEBUG 2018/08/14 12:30:04.250 58830 Start event\*  
... and so on

From this log i want to have one event with tags like  
UID =\> uniq id  
EventStart =\> 2018/08/14 12:30:03.389  
ProcTime =\> 0.005 sec

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [October 20, 2018, 10:18am UTC](https://discuss.elastic.co/t/parsing-multiline-log/149539/2 "2018-10-20T10:18:28Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
