# Multiline log

**URL:** https://discuss.elastic.co/t/multiline-log/43075
**Category:** Logstash
**Created:** [March 1, 2016, 5:48am UTC](https://discuss.elastic.co/t/multiline-log/43075 "2016-03-01T05:48:56Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![pecwich](https://avatars.discourse-cdn.com/v4/letter/p/bc79bd/32.png) [@pecwich](https://discuss.elastic.co/u/pecwich)
#### Post date: [March 1, 2016, 5:48am UTC](https://discuss.elastic.co/t/multiline-log/43075/1 "2016-03-01T05:48:57Z")

</div>

In a multiline log, can parsing or indexing of fields occur only in the first line? I want to parse fields present in the second or third line for example, what should I do?

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [March 3, 2016, 7:05am UTC](https://discuss.elastic.co/t/multiline-log/43075/2 "2016-03-03T07:05:19Z")

</div>

That works but you may have to put `(?m)` at the beginning of the grok expression (which I assume is what you're using). For specific help please post an example message and your configuration.

---

<div class="post-metadata">

### Author: ![pecwich](https://avatars.discourse-cdn.com/v4/letter/p/bc79bd/32.png) [@pecwich](https://discuss.elastic.co/u/pecwich)
#### Post date: [March 4, 2016, 6:01am UTC](https://discuss.elastic.co/t/multiline-log/43075/3 "2016-03-04T06:01:59Z")

</div>

Like this is my input{

file {  
path =\> "/home/.\*.com"  
type =\> "framework"  
codec =\> multiline {

```
pattern => 

```

"(?(?:Mon(?:day)?|Tue(?:sday)?|Wed(?:nesday)?|Thu(?:rsday)?|Fri(?:day)?|Sat(?:urday)?|Sun(?:day)?)  
%{MONTH} (?:(?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9])  
%{HOUR}:%{MINUTE}:%{SECOND} %{YEAR})"  
negate =\> true  
what =\> "previous"  
}  
}

and this is my filter:  
grok{  
match =\> [ "message" ,  
"(?(?:Mon(?:day)?|Tue(?:sday)?|Wed(?:nesday)?|Thu(?:rsday)?|Fri(?:day)?|Sat(?:urday)?|Sun(?:day)?)  
%{MONTH} (?:(?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9])  
%{HOUR}:%{MINUTE}:%{SECOND} %{YEAR}) %{GREEDYDATA}  
(ID=:)(?[0-9]{2,3}.[0-9]{2,3}.[0-9]{2,3}.[0-9]{2,3}([0-9]{1}))" ]  
}

And suppose this is my log:

Thu Feb 25 15:00:03 2016 nbdnclmlmc;lm [INFO] kjnclkklmc  
nckdnlmvl;  
nkcjlkmcp;  
nmdmc;c; RID=:10.88.81.211(0)  
jhcbdkjnclkasdncl

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [March 4, 2016, 6:38am UTC](https://discuss.elastic.co/t/multiline-log/43075/4 "2016-03-04T06:38:06Z")

</div>

Always always always post configurations (especially those with regular expressions) formatted as code! Your snippet above is not copy/pasteable and I had to rewrite it just to get it past the syntax check. Because I don't know exactly what you have I don't know exactly what was wrong with it, but the following works:

```auto
$ cat test.config 
input {
  stdin {
    codec => multiline {
      pattern => "%{DAY} %{MONTH} %{MONTHDAY} %{TIME} %{YEAR}"
      negate => true
      what => "previous"
    }
  }
}
output { stdout { codec => rubydebug } }
filter {
  grok {
    match => [
      "message",
      "%{DAY} %{MONTH} %{MONTHDAY} %{TIME} %{YEAR} %{GREEDYDATA}; RID=:%{IP:ip}"
    ]
  }
}
$ cat data 
Thu Feb 25 15:00:03 2016 nbdnclmlmc;lm [INFO] kjnclkklmc
nckdnlmvl;
nkcjlkmcp;
nmdmc;c; RID=:10.88.81.211(0)
jhcbdkjnclkasdncl
Thu Feb 25 15:00:03 2016 nbdnclmlmc;lm [INFO] kjnclkklmc
nckdnlmvl;
nkcjlkmcp;
nmdmc;c; RID=:10.88.81.211(0)
jhcbdkjnclkasdncl
$ /opt/logstash/bin/logstash -f test.config < data
Settings: Default pipeline workers: 8
Logstash startup completed
{
    "@timestamp" => "2016-03-04T06:37:33.464Z",
       "message" => "Thu Feb 25 15:00:03 2016 nbdnclmlmc;lm [INFO] kjnclkklmc\nnckdnlmvl;\nnkcjlkmcp;\nnmdmc;c; RID=:10.88.81.211(0)\njhcbdkjnclkasdncl",
      "@version" => "1",
          "tags" => [
        [0] "multiline"
    ],
          "host" => "lnxolofon",
            "ip" => "10.88.81.211"
}
Logstash shutdown completed

```

---

<div class="post-metadata">

### Author: ![pecwich](https://avatars.discourse-cdn.com/v4/letter/p/bc79bd/32.png) [@pecwich](https://discuss.elastic.co/u/pecwich)
#### Post date: [March 4, 2016, 9:09am UTC](https://discuss.elastic.co/t/multiline-log/43075/5 "2016-03-04T09:09:09Z")

</div>

GREEDYDATA does not capture the new line character, right? So how did it parse the RID?  
You didn't even use (?m).

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [March 4, 2016, 9:11am UTC](https://discuss.elastic.co/t/multiline-log/43075/6 "2016-03-04T09:11:57Z")

</div>

I would've expected `(?m)` to be necessary but I don't remember its exact semantics.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 5:08am UTC](https://discuss.elastic.co/t/multiline-log/43075/7 "2017-07-06T05:08:23Z")

</div>


