# Extract log between two patterns

**URL:** https://discuss.elastic.co/t/extract-log-between-two-patterns/152426
**Category:** Logstash
**Created:** [October 15, 2018, 5:48am UTC](https://discuss.elastic.co/t/extract-log-between-two-patterns/152426 "2018-10-15T05:48:29Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![Karanth](https://avatars.discourse-cdn.com/v4/letter/k/a88e57/32.png) [@Karanth](https://discuss.elastic.co/u/Karanth)
#### Post date: [October 15, 2018, 5:48am UTC](https://discuss.elastic.co/t/extract-log-between-two-patterns/152426/1 "2018-10-15T05:48:29Z")

</div>

Hi

We have logs that are very huge. I want to be able to extract logs [events] between two patterns.

For example:  
2018/08/29 14:33:54.646 - some log  
2018/08/29 14:33:58.809 - pattern1 -  
rest of the log message under pattern 1  
2018/08/29 14:30:58.338 - some log msg  
2018/08/29 14:33:58.809 - pattern2 -  
rest of the log message under pattern 2

I would like to extract -  
2018/08/29 14:33:58.809 - pattern1 -  
rest of the log message under pattern 1  
2018/08/29 14:30:58.338 - some log msg  
2018/08/29 14:33:58.809 - pattern2 -  
rest of the log message under pattern 2

I know to drop the message if a pattern is matched but I do not know to extract all the events between two matching patterns. Can someone please help me with this? Thank you

---

<div class="post-metadata">

### Author: ![Makra](https://avatars.discourse-cdn.com/v4/letter/m/8491ac/32.png) [@Makra](https://discuss.elastic.co/u/Makra)
#### Post date: [October 15, 2018, 2:34pm UTC](https://discuss.elastic.co/t/extract-log-between-two-patterns/152426/2 "2018-10-15T14:34:29Z")

</div>

You can have more than one patterns in the filter section so that only matching pattern will be parsed and rest of event will have `_grokparsefailure` that you can drop.

```
filter {
          grok {
		          match=>[
				            "message","%{pattern1}",
                            "message","%{pattern2}",
                            "message","%{pattern3}"
						 ]
		  }
		  if "_grokparsefailure" in [tags] {
                  drop { }
          }		

}
```

---

<div class="post-metadata">

### Author: ![Karanth](https://avatars.discourse-cdn.com/v4/letter/k/a88e57/32.png) [@Karanth](https://discuss.elastic.co/u/Karanth)
#### Post date: [October 16, 2018, 4:42am UTC](https://discuss.elastic.co/t/extract-log-between-two-patterns/152426/3 "2018-10-16T04:42:58Z")

</div>

Thanks for replying.  
With this, I can capture events matching pattern1 and pattern2 but I also need the events that fall **between pattern1 and pattern2**. There are no specific patterns for those events. So am unable to match those, am just able to extract events that match pattern1 and pattern2.

---

<div class="post-metadata">

### Author: ![hfakar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hfakar/32/18211_2.png) [@hfakar](https://discuss.elastic.co/u/hfakar)
#### Post date: [October 16, 2018, 8:43am UTC](https://discuss.elastic.co/t/extract-log-between-two-patterns/152426/4 "2018-10-16T08:43:57Z")

</div>

You can write these pattern1 pattern2 in if statements just like in \_grokparsefailiure. And a general filter above. The ones that doesnt fall into ifs will fall into that general filter. Thats how "else" logic works in logstash I guess.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 13, 2018, 8:43am UTC](https://discuss.elastic.co/t/extract-log-between-two-patterns/152426/5 "2018-11-13T08:43:59Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
