# How to parse fields from multiline text log file?

**URL:** <https://discuss.elastic.co/t/how-to-parse-fields-from-multiline-text-log-file/265752>\
**Category:** Logstash\
**Created:** [March 1, 2021, 6:14am UTC](https://discuss.elastic.co/t/how-to-parse-fields-from-multiline-text-log-file/265752 "2021-03-01T06:14:34Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![d.silwon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/d.silwon/32/65853_2.png) [@d.silwon](https://discuss.elastic.co/u/d.silwon)\
**Post date:** [March 1, 2021, 6:14am UTC](https://discuss.elastic.co/t/how-to-parse-fields-from-multiline-text-log-file/265752/1 "2021-03-01T06:14:34Z")

</div>

Dears,

I need your advise in case of parsing fields from text log file. This is sample of text file:

```auto
---- got a mess at: 11:21:51
ctrl_mess_handler::control_message_handler @333.333
got new timer id: 11111, to be fired off in: 2 seconds
ip_timer::cc_timer_expd @333.333
:profiler: 0
setting next timer event in 2 seconds
wa at 11:21:51, q id: 44444
ip_timer::cc_timer_expd @112156.1300527189
sending timer expired msg to q: 98323 for cc_con
SENT OK: timer expired msg to q: 98323, type=0, type2=0
:profiler - wa end: 111
ccad_timer mess receive interrupted, errno = EINTR (sleep on a full mess q condition, the process caught a signal)
:profiler: 111
setting next timer event in 7 seconds
wa at 11:21:56, q id: 44444
:profiler - wa end: 1

```

What is the best way to extract some fields to separate columns? How to do it?  
For example I need these below fields in separate columns in ELK:  
"got new timer id: 11111"  
"q id: 44444"  
"type=0"  
"wa end: 1"  
Best Regrads,  
Dan

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [March 1, 2021, 5:53pm UTC](https://discuss.elastic.co/t/how-to-parse-fields-from-multiline-text-log-file/265752/2 "2021-03-01T17:53:47Z")

</div>

You could use grok

```
grok {
    break_on_match => false
    match => {
        "message" => [
            "got new timer id: %{INT:timerId}",
            "q id: %{INT:qId}",
            "type=%{INT:type}",
            "wa end: %{INT:waEnd}"
        ]
    }
}
```

---

<div class="post-metadata">

**Author:** ![d.silwon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/d.silwon/32/65853_2.png) [@d.silwon](https://discuss.elastic.co/u/d.silwon)\
**Post date:** [March 1, 2021, 6:22pm UTC](https://discuss.elastic.co/t/how-to-parse-fields-from-multiline-text-log-file/265752/3 "2021-03-01T18:22:14Z")

</div>

> [@Badger](#):
>
> `wa end: %{INT:waEnd}`

Hello @Badger ,

On the base of your answer I prepared some grok patter:

```auto
(?m)%{TIME}%{GREEDYDATA}got new timer id: %{INT:timerId}%{GREEDYDATA}q id: %{INT:qId}%{GREEDYDATA}type=%{INT:type}%{GREEDYDATA}wa end: %{INT:waEnd}

```

and it looks good. I'll test your proposal too. Thanks a lot.

Best Regards,  
Dan

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [March 1, 2021, 8:02pm UTC](https://discuss.elastic.co/t/how-to-parse-fields-from-multiline-text-log-file/265752/4 "2021-03-01T20:02:42Z")

</div>

Having a pattern with several GREEDYDATA embedded is going to be more expensive than using multiple patterns. Maybe not expensive enough to matter, but at least a little.

---

<div class="post-metadata">

**Author:** ![d.silwon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/d.silwon/32/65853_2.png) [@d.silwon](https://discuss.elastic.co/u/d.silwon)\
**Post date:** [March 2, 2021, 4:27am UTC](https://discuss.elastic.co/t/how-to-parse-fields-from-multiline-text-log-file/265752/5 "2021-03-02T04:27:45Z")

</div>

Hello @Badger,

Thanks a lot for your explanation. I'll test your proposed sollution.

Best Regards,  
Dan

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 30, 2021, 4:28am UTC](https://discuss.elastic.co/t/how-to-parse-fields-from-multiline-text-log-file/265752/6 "2021-03-30T04:28:09Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.