# Multiline issue with log

**URL:** <https://discuss.elastic.co/t/multiline-issue-with-log/140598>\
**Category:** Logstash\
**Created:** [July 18, 2018, 5:57pm UTC](https://discuss.elastic.co/t/multiline-issue-with-log/140598 "2018-07-18T17:57:47Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![AGarcMan](https://avatars.discourse-cdn.com/v4/letter/a/919ad9/32.png) [@AGarcMan](https://discuss.elastic.co/u/AGarcMan)\
**Post date:** [July 18, 2018, 5:57pm UTC](https://discuss.elastic.co/t/multiline-issue-with-log/140598/1 "2018-07-18T17:57:47Z")

</div>

Hi,

I'm working with a log with the following structure (fragment):

```
  2018-01-30 18:14:10.466554
  (0x03000000:PCDataField ):folio = '201711231427280265' (CHARACTER)
  (0x03000000:PCDataField ):idProceso = '7' (CHARACTER)
  (0x03000000:PCDataField ):idSubproceso = '101' (CHARACTER)
  (0x03000000:PCDataField ):idSubetapa = '273' (CHARACTER)
  (0x03000000:PCDataField ):idSnapshot = '2064.c76a4df4-7e11-4c0a-9a04-f305faaef4d6' (CHARACTER)
  (0x01000000:Folder ):parametros = (
    (0x01000000:Folder):parametro = (
      (0x03000000:PCDataField):nombre = 'REPROCESO' (CHARACTER)
      (0x01000000:Folder ):valores = (
        (0x03000000:PCDataField):valor = '2' (CHARACTER)
      )
    )
    (0x01000000:Folder):parametro = (
      (0x03000000:PCDataField):nombre = 'ID_INSTANCIA' (CHARACTER)
      (0x01000000:Folder ):valores = (
        (0x03000000:PCDataField):valor = '16649' (CHARACTER)
      )
    )....

```

each time a new event is generated a similar block is added to the log, I filter this new event using a codec multiline like this:

input {  
file {  
path =\> "/home/test.log"  
start\_position =\> "beginning"  
codec =\> multiline {  
pattern =\> "^%{TIMESTAMP\_ISO8601} "  
negate =\> true  
what =\> "previous"  
}  
}  
}

Then when i try to get the fields (folio, idProceso, etc), using the following filter:

grok{  
match =\> { "message" =\> "\W\w+:\w+\W:%{DATA:linea}\(" }  
}

It only applies the filter to the first line of the multiline, for example it only returns: folio = '201711231427280265'

I don't know, how to change the filter to apply it again to the remaining lines of the log?

Thanks for your help

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [July 18, 2018, 6:12pm UTC](https://discuss.elastic.co/t/multiline-issue-with-log/140598/2 "2018-07-18T18:12:42Z")

</div>

This category is for discussions in English. If you want to post in Spanish please use [https://discuss.elastic.co/c/in-your-native-tongue/elastic-en-espanol](https://discuss.elastic.co/c/in-your-native-tongue/elastic-en-espanol).

---

<div class="post-metadata">

**Author:** ![AGarcMan](https://avatars.discourse-cdn.com/v4/letter/a/919ad9/32.png) [@AGarcMan](https://discuss.elastic.co/u/AGarcMan)\
**Post date:** [July 18, 2018, 6:26pm UTC](https://discuss.elastic.co/t/multiline-issue-with-log/140598/3 "2018-07-18T18:26:03Z")

</div>

Ok, thanks

---

<div class="post-metadata">

**Author:** ![AGarcMan](https://avatars.discourse-cdn.com/v4/letter/a/919ad9/32.png) [@AGarcMan](https://discuss.elastic.co/u/AGarcMan)\
**Post date:** [July 18, 2018, 6:57pm UTC](https://discuss.elastic.co/t/multiline-issue-with-log/140598/4 "2018-07-18T18:57:24Z")

</div>

I changed the post to English 🙂

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [July 18, 2018, 7:12pm UTC](https://discuss.elastic.co/t/multiline-issue-with-log/140598/5 "2018-07-18T19:12:37Z")

</div>

That grok filter is not valid (it has a trailing parenthesis). Can you fix that?

If I had to parse that I would go after it with something like this

```
    grok{
        break_on_match => false
        match => { "message" => [ 
                "\A%{DATA:ts}
",
                ":folio\s+= '%{DATA:folio}'",
                ":idProceso\s+= '%{DATA:idProceso}'",
                ":idSubproceso\s+= '%{DATA:idSubproceso}'",
                ":idSubetapa\s+= '%{DATA:idSubetapa}'",
                ":idSnapshot\s+= '%{DATA:idSnapshot}'"
            ]
        }
    }

```

Plus maybe this, although it is ugly and fragile.

```
    ruby {
        code => "
            m = event.get('message')
            m1 = m.scan( /nombre\s+= '(?<nombre>[^']+)'/ )
            m2 = m.scan( /valor = '(?<valor>[^']+)'/ )
            h = {}
            m1.to_a.each_index { |i|
                h[m1[i][0].to_s ] = m2[i][0].to_s
            }
            event.set('parametros', h)
        "   
    }
```

---

<div class="post-metadata">

**Author:** ![AGarcMan](https://avatars.discourse-cdn.com/v4/letter/a/919ad9/32.png) [@AGarcMan](https://discuss.elastic.co/u/AGarcMan)\
**Post date:** [July 18, 2018, 10:05pm UTC](https://discuss.elastic.co/t/multiline-issue-with-log/140598/6 "2018-07-18T22:05:32Z")

</div>

Thank you so much, it worked perfectly for my problem.

I have limited experience in ELK,it would be great if you could recommend some pages or books where I can learn more about ELK?

Looking forward to hearing from you.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 15, 2018, 10:05pm UTC](https://discuss.elastic.co/t/multiline-issue-with-log/140598/7 "2018-08-15T22:05:35Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.