# How to get all matches for a grok pattern in a multiline message

**URL:** <https://discuss.elastic.co/t/how-to-get-all-matches-for-a-grok-pattern-in-a-multiline-message/181>\
**Category:** Logstash\
**Created:** [May 5, 2015, 7:01am UTC](https://discuss.elastic.co/t/how-to-get-all-matches-for-a-grok-pattern-in-a-multiline-message/181 "2015-05-05T07:01:41Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![markus](https://avatars.discourse-cdn.com/v4/letter/m/b782af/32.png) [@markus](https://discuss.elastic.co/u/markus)\
**Post date:** [May 5, 2015, 7:01am UTC](https://discuss.elastic.co/t/how-to-get-all-matches-for-a-grok-pattern-in-a-multiline-message/181/1 "2015-05-05T07:01:41Z")

</div>

Hi,  
I am having a multiline log format that consists of an XML document with a very deep message structure. All elements in the data structure can have a child element called Message containing a value attribute like so:

```
<ElementA>
    <ElementB>
         <ElementC>
              <Message value="foo"/>
        </ElementC>
    </ElementB>
    <ElementB>
         <ElementC/>
    </ElementB>
    <ElementB>
         <ElementC/>
         <Message value="bar"/>
    </ElementB>
    <Message value="even more bar"/>
</ElementA>

```

This means the number of elements varies from message to message and can get quite large. What I would like to do is to build a Kibana table showing all message values with their respective number of occurences.

To do so I thought I can extract an array of all value attributes in my multiline message with a pattern like:

```
    grok {
        break_on_match => false
        match => ["message","<Message value=\"%{DATA:msgText}"]
    }

```

However grok only finds the first match for my pattern. If it is not possible to get all matches, is it possible to get the first n matches?  
If this is not possible using logstash filtering can I get something like that done on the elasticsearch/kibana side?

Any help appreciated.  
Bye,  
Markus

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [May 5, 2015, 10:05am UTC](https://discuss.elastic.co/t/how-to-get-all-matches-for-a-grok-pattern-in-a-multiline-message/181/2 "2015-05-05T10:05:55Z")

</div>

Can't you use the [xml filter](http://logstash.net/docs/1.4.2/filters/xml)? It looks like the xpath parameter should do exactly what you want:

> Values returned by XPath parsring [sic] from xpath-synatx [sic] will be put in the destination field. Multiple values returned will be pushed onto the destination field as an array.

---

<div class="post-metadata">

**Author:** ![markus](https://avatars.discourse-cdn.com/v4/letter/m/b782af/32.png) [@markus](https://discuss.elastic.co/u/markus)\
**Post date:** [May 5, 2015, 4:26pm UTC](https://discuss.elastic.co/t/how-to-get-all-matches-for-a-grok-pattern-in-a-multiline-message/181/3 "2015-05-05T16:26:11Z")

</div>

Thanks Magnus,  
works like a treat. You saved my day.

---

<div class="post-metadata">

**Author:** ![Shaun\_Wells](https://avatars.discourse-cdn.com/v4/letter/s/b487fb/32.png) [@Shaun\_Wells](https://discuss.elastic.co/u/Shaun_Wells)\
**Post date:** [May 27, 2015, 3:07pm UTC](https://discuss.elastic.co/t/how-to-get-all-matches-for-a-grok-pattern-in-a-multiline-message/181/4 "2015-05-27T15:07:44Z")

</div>

Any chance of seeing what your config looks like when using the XML Filter as I'm having a few issues myself..

---

<div class="post-metadata">

**Author:** ![markus](https://avatars.discourse-cdn.com/v4/letter/m/b782af/32.png) [@markus](https://discuss.elastic.co/u/markus)\
**Post date:** [May 29, 2015, 9:13am UTC](https://discuss.elastic.co/t/how-to-get-all-matches-for-a-grok-pattern-in-a-multiline-message/181/5 "2015-05-29T09:13:47Z")

</div>

Hi Shaun,  
my config for the xml filter is pretty simple. I use it like this with an XPATH to filter the parts I'm interested in.

```
    xml {
        source => "message"
        target => doc
        store_xml => false
        xpath => ["//Message[@id!=0]","msgs"]
    }
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:39am UTC](https://discuss.elastic.co/t/how-to-get-all-matches-for-a-grok-pattern-in-a-multiline-message/181/6 "2017-07-06T05:39:04Z")

</div>