# Need to parse multi-lines log message

**URL:** <https://discuss.elastic.co/t/need-to-parse-multi-lines-log-message/360206>\
**Category:** Logstash\
**Created:** [May 25, 2024, 1:40am UTC](https://discuss.elastic.co/t/need-to-parse-multi-lines-log-message/360206 "2024-05-25T01:40:04Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![iceman0410](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/iceman0410/32/134395_2.png) [@iceman0410](https://discuss.elastic.co/u/iceman0410)\
**Post date:** [May 25, 2024, 1:40am UTC](https://discuss.elastic.co/t/need-to-parse-multi-lines-log-message/360206/1 "2024-05-25T01:40:04Z")

</div>

Hi everyone,

I am new on Grok. I am in stuck with multiple lines of log message below. Can anyone help me look at my grok. Thanks  
The log message:  
[timestamp: 1621431760] abort handler of pid 1823 thread 1848977280  
\*\*\* Stacks of threads \*\*\* (current thread is 1848977280)  
Stack of thread=1848977280, depth=3  
main  
shutdownServices  
EMThriftServer::stop

logstash show

```
 "@version" => "1",
"@timestamp" => 2024-05-25T01:32:29.912Z,
     "event" => {
    "original" => " Stack of thread=1848977280, depth=3"
},
       "ecs" => {
    "version" => "8.0.0"
},
      "tags" => [
    [0] "beats_input_codec_plain_applied",
    **[1] "_grokparsefailure"**

```

I tried: filter  
{  
grok {  
match =\> { "message" =\> "[%{WORD}:%{SPACE}%{NUMBER:dts}]%{SPACE}(\<?rest\>%{GREEDYDATA}(?m))" }  
}  
date {  
match =\> ["dts", "ddHHmmssSSS"]  
target =\> "@timestamp"  
timezone =\> "UTC"  
}

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [May 25, 2024, 2:09am UTC](https://discuss.elastic.co/t/need-to-parse-multi-lines-log-message/360206/2 "2024-05-25T02:09:10Z")

</div>

logstash is consuming the multi-line log message one line at a time. If you are using filebeat you should use the multiline processing there to combine the lines into a single event.

---

<div class="post-metadata">

**Author:** ![iceman0410](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/iceman0410/32/134395_2.png) [@iceman0410](https://discuss.elastic.co/u/iceman0410)\
**Post date:** [May 25, 2024, 2:15am UTC](https://discuss.elastic.co/t/need-to-parse-multi-lines-log-message/360206/3 "2024-05-25T02:15:41Z")

</div>

Hi, this is my filebeat.yml. Can you advice if it is not correct.Thanks

filebeat.inputs:

# filestream is an input for collecting log messages from files.

- type: log

---

<div class="post-metadata">

**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\
**Post date:** [May 25, 2024, 1:19pm UTC](https://discuss.elastic.co/t/need-to-parse-multi-lines-log-message/360206/4 "2024-05-25T13:19:32Z")

</div>

You can use something like this:

filebeat.yml

```auto
- type: filestream
  id: idlog
  enabled: true
  paths:
    - C:\Filebeat\logs*.txt
  parsers:
  - multiline:
      type: pattern
      pattern: '^\[timestamp:'
      negate: true
      match: after

```

logstash.conf contains extracted pid, thread and 2 part of message. I don't know a logic, which fields are important. Easiest is to put everything in the message after the timestamp. Of course you can parse depth, I assume services as fields.

```auto
input {
  beats {
    port => 5044
  }  
}

filter {

	grok {match => {
	message => "\[timestamp: %{POSINT:timestamp}\]%{SPACE}%{DATA:msgpart1}%{SPACE}pid %{POSINT:pid} thread %{POSINT:threadid}%{GREEDYDATA:msgpart2}"
		}    
	}
	date {
		match => ["timestamp", "UNIX"]
	}

}

output {
    stdout { codec => rubydebug{} }
}

```

Result:

```auto
{
      "threadid" => "1848977281",
     "timestamp" => "1621431765",
       "message" => "[timestamp: 1621431765] abort handler of pid 1111 thread 1848977281\n ***Stacks of threads*** (current thread is 1848977281)\nStack of thread=1848977281, depth=1\nmain\nshutdownServices\nEMThriftServer::stop",
    "@timestamp" => 2021-05-19T13:42:45.000Z,
      "@version" => "1",
           "pid" => "1111",
      "msgpart2" => "\n ***Stacks of threads*** (current thread is 1848977281)\nStack of thread=1848977281, depth=1\nmain\nshutdownServices\nEMThriftServer::stop",
      "msgpart1" => "abort handler of"
}

```

---

<div class="post-metadata">

**Author:** ![iceman0410](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/iceman0410/32/134395_2.png) [@iceman0410](https://discuss.elastic.co/u/iceman0410)\
**Post date:** [May 27, 2024, 10:41am UTC](https://discuss.elastic.co/t/need-to-parse-multi-lines-log-message/360206/5 "2024-05-27T10:41:30Z")

</div>

thanks @Rios and @Badger