# Pattern Matching

**URL:** <https://discuss.elastic.co/t/pattern-matching/77195>\
**Category:** Logstash\
**Created:** [March 2, 2017, 5:31pm UTC](https://discuss.elastic.co/t/pattern-matching/77195 "2017-03-02T17:31:47Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![cisaksen](https://avatars.discourse-cdn.com/v4/letter/c/49beb7/32.png) [@cisaksen](https://discuss.elastic.co/u/cisaksen)\
**Post date:** [March 2, 2017, 5:31pm UTC](https://discuss.elastic.co/t/pattern-matching/77195/1 "2017-03-02T17:31:47Z")

</div>

Using grok I want to parse a log file as you normally would but i also want to pull a specific piece of of the log record in a custom field. I have 2 patterns one is a nginx\_combined with a x-forwarded field at the end. The second is the specific piece of the log that i'm trying to put into it own field. ? (?i)((http[s]?)(://.+?)/)

I've tried this but it isn't creating the custom field.

> grok {  
> patterns\_dir =\> ["/etc/logstash/patterns"]  
> break\_on\_match =\> false  
> match =\> { "message" =\> "%{SED\_NGINX\_COMBINE}" }  
> match =\> { "message" =\> "%{SED\_HTTPHOST:httphost}" }   
> add\_tag =\> ["drupal-staging"]  
> }

Basically I want both patterns to exist in a single record in elasticsearch. I first though that an addfield would work but is was suggested that i try it this way. Any ideas ?

---

<div class="post-metadata">

**Author:** ![glmrenard](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/glmrenard/32/15393_2.png) [@glmrenard](https://discuss.elastic.co/u/glmrenard)\
**Post date:** [March 3, 2017, 2:35pm UTC](https://discuss.elastic.co/t/pattern-matching/77195/2 "2017-03-03T14:35:38Z")

</div>

Hello

You can do something like  
`match => { "message" => "<182>%{WORD:program}: %{COMBINEDAPACHELOG} \"((?<x_forwarded_for>%{IP:xff_clientip}, .*)|-)\" %{NUMBER:request_time:integer} %{IPORHOST:targethost}"}`

Or using  
`=> "(%{SED_NGINX_COMBINE})|(%{SED_HTTPHOST:httphost}))"`  
Regards, Guillaume

---

<div class="post-metadata">

**Author:** ![cisaksen](https://avatars.discourse-cdn.com/v4/letter/c/49beb7/32.png) [@cisaksen](https://discuss.elastic.co/u/cisaksen)\
**Post date:** [March 7, 2017, 6:21pm UTC](https://discuss.elastic.co/t/pattern-matching/77195/3 "2017-03-07T18:21:41Z")

</div>

with this method what does the '|' do ?

=\> "(%{SED\_NGINX\_COMBINE})|(%{SED\_HTTPHOST:httphost}))"

Does it cat them or is it a "or"

---

<div class="post-metadata">

**Author:** ![glmrenard](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/glmrenard/32/15393_2.png) [@glmrenard](https://discuss.elastic.co/u/glmrenard)\
**Post date:** [March 7, 2017, 6:56pm UTC](https://discuss.elastic.co/t/pattern-matching/77195/4 "2017-03-07T18:56:49Z")

</div>

It's an **or**

---

<div class="post-metadata">

**Author:** ![cisaksen](https://avatars.discourse-cdn.com/v4/letter/c/49beb7/32.png) [@cisaksen](https://discuss.elastic.co/u/cisaksen)\
**Post date:** [March 7, 2017, 9:37pm UTC](https://discuss.elastic.co/t/pattern-matching/77195/5 "2017-03-07T21:37:43Z")

</div>

Ok - unfortunately that's not what I'm trying to do. I'm trying to parse the same section of the log record 2 different ways and have both in the elasticsearch record as 2 different fields.

Particularly the referrer field. The first as %{QS:referrer} and then just part of it in a different field from this pattern: (?i)((http[s]?)(://.+?)/)

There doesn't seem to be a easy way to do this.

---

<div class="post-metadata">

**Author:** ![glmrenard](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/glmrenard/32/15393_2.png) [@glmrenard](https://discuss.elastic.co/u/glmrenard)\
**Post date:** [March 8, 2017, 7:02am UTC](https://discuss.elastic.co/t/pattern-matching/77195/6 "2017-03-08T07:02:43Z")

</div>

Hi,  
Then maybe you should cheat, by example with something like  
`"message" => "%{GREEDYDATA:message2}"}`  
And then do two filter

- on message with your first filter
- on message2 (which is a clone of message) with the second one

---

<div class="post-metadata">

**Author:** ![cisaksen](https://avatars.discourse-cdn.com/v4/letter/c/49beb7/32.png) [@cisaksen](https://discuss.elastic.co/u/cisaksen)\
**Post date:** [March 8, 2017, 1:21pm UTC](https://discuss.elastic.co/t/pattern-matching/77195/7 "2017-03-08T13:21:55Z")

</div>

ok i'll try that. Thanks

There isn't a reference somewhere for grok key words is there. I've been looking but can't seem to find one that lists and explains the pattern key words i see on git hub and other examples like your own. For example the WORD, QUOTEDSTRING, NUMBER, GREEDYDATA and many others ? I just would like to know how they are defined.

Thanks again

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [March 8, 2017, 1:30pm UTC](https://discuss.elastic.co/t/pattern-matching/77195/8 "2017-03-08T13:30:12Z")

</div>

The definition of the patterns can be found [here](https://github.com/logstash-plugins/logstash-patterns-core/tree/master/patterns). The ones you mentioned [are defined in this file](https://github.com/logstash-plugins/logstash-patterns-core/blob/master/patterns/grok-patterns).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 5, 2017, 1:30pm UTC](https://discuss.elastic.co/t/pattern-matching/77195/9 "2017-04-05T13:30:20Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
