# Regular expression problem

**URL:** https://discuss.elastic.co/t/regular-expression-problem/26484
**Category:** Logstash
**Created:** [July 29, 2015, 1:09pm UTC](https://discuss.elastic.co/t/regular-expression-problem/26484 "2015-07-29T13:09:14Z")
**Posts on this page:** 13
**Page:** 1

<div class="post-metadata">

### Author: ![simonrisberg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simonrisberg/32/3513_2.png) [@simonrisberg](https://discuss.elastic.co/u/simonrisberg)
#### Post date: [July 29, 2015, 1:09pm UTC](https://discuss.elastic.co/t/regular-expression-problem/26484/1 "2015-07-29T13:09:14Z")

</div>

Hi!

I'm trying to make a regular expression to get out a certain word from a URIPATH that is not actually in the parh itself. So far I've actually managed to do that although when I put it into my logstash configuration it doesn't like the syntax so it "gracefully" stops ELK from starting up. I know that my pattern is correct because I've tried it with a grok debugger.

Typical event message:10.67.6.51 - - [21/Jun/2015:21:14:21 +0000] "GET /nexus/content/repositories/jts-development/com/jeppesen/jcms/maven-metadata.xml.sha1 HTTP/1.1" 200 40

My expression: "(?[^/]+) /nexus/content/repositories/"

What shows up in the grok debugger: "GET"

How my logstash configuration looks: (it's the last pattern in the grok filter)

```
grok {

     type => "nexus-log"
     break_on_match => false

     match => [
        "message", "\b\w+\b\s/nexus/content/repositories/(?<repositories>[^/]+)",
        "message", "(?<mytimestamp>%{MONTHDAY}/%{MONTH}/%{YEAR}:%{HOUR}:%{MINUTE}:%{SECOND} %{ISO8601_TIMEZONE})",
        "message", " "(?<requesttype>[^/]+) /nexus/content/repositories/"
      ]
   }
```

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [July 29, 2015, 3:14pm UTC](https://discuss.elastic.co/t/regular-expression-problem/26484/2 "2015-07-29T15:14:16Z")

</div>

- If you want a double quote inside your expression you need to escape it with a backslash. That's most likely why Logstash doesn't start.
- I can only assume that this expression results in a trailing space at the end of the resulting `requesttype` field. Why not just use %{WORD:requesttype} to match the HTTP method? They never contain spaces anyway.
- It would've been way easier to just use the predefined grok pattern for this kind of logfile (it looks like an Apache common file) to get everything into separate fields without any custom expressions at all.

---

<div class="post-metadata">

### Author: ![simonrisberg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simonrisberg/32/3513_2.png) [@simonrisberg](https://discuss.elastic.co/u/simonrisberg)
#### Post date: [July 30, 2015, 7:31am UTC](https://discuss.elastic.co/t/regular-expression-problem/26484/3 "2015-07-30T07:31:18Z")

</div>

Thank you, I'm gonna try to use the WORD pattern. If that doesn't work, where should I insert the backslash?

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [July 30, 2015, 7:33am UTC](https://discuss.elastic.co/t/regular-expression-problem/26484/4 "2015-07-30T07:33:46Z")

</div>

Use the backslash to escape double quotes that occur within the regular expressions. Or, you could make the regular expression single-quoted (i.e. it's delimited by single quotes rather than double quotes).

---

<div class="post-metadata">

### Author: ![simonrisberg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simonrisberg/32/3513_2.png) [@simonrisberg](https://discuss.elastic.co/u/simonrisberg)
#### Post date: [July 30, 2015, 8:14am UTC](https://discuss.elastic.co/t/regular-expression-problem/26484/5 "2015-07-30T08:14:01Z")

</div>

Thanks for the help. The predefined pattern worked just fine. Although I have a new problem that has risen. When the pattern doesn't succeed in matching anything on certain events which is correct because it shouldn't it still shows some kind of result but it becoems a "-". Is there anyway to get rid of that? I'm guessing it's some kind of grokparsefailure?

Picture below.

 ![](https://us1.discourse-cdn.com/elastic/original/2X/7/7f320f8fd4bd4641a848807faa04dd8895ced5dc.jpg)

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [July 30, 2015, 1:15pm UTC](https://discuss.elastic.co/t/regular-expression-problem/26484/6 "2015-07-30T13:15:36Z")

</div>

What do those messages look like in full and what's your filter configuration?

---

<div class="post-metadata">

### Author: ![simonrisberg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simonrisberg/32/3513_2.png) [@simonrisberg](https://discuss.elastic.co/u/simonrisberg)
#### Post date: [July 30, 2015, 1:39pm UTC](https://discuss.elastic.co/t/regular-expression-problem/26484/7 "2015-07-30T13:39:06Z")

</div>

The message in full looks like this

 ![](https://us1.discourse-cdn.com/elastic/original/2X/7/72a041684ce88f66ca7ddf65561c393cba6219b1.jpg)

My filter configuration looks like this

```
filter {

   grok {

     type => "nexus-log"
     break_on_match => false

     match => [
        "message", "\b\w+\b\s/nexus/content/repositories/(?<repositories>[^/]+)",
        "message", "(?<mytimestamp>%{MONTHDAY}/%{MONTH}/%{YEAR}:%{HOUR}:%{MINUTE}:%{SECOND} %{ISO8601_TIMEZONE})",
        "message", "(%{WORD:requesttype}) /nexus/content/repositories/"
      ]
   }
   date{
      match => ["mytimestamp", "dd/MMM/YYYY:HH:mm:ss Z"]
      remove_field => ["mytimestamp"]
   }

}

```

Note that this is nothing that really needs an urgent fix although it would look nicer.

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [July 30, 2015, 2:39pm UTC](https://discuss.elastic.co/t/regular-expression-problem/26484/8 "2015-07-30T14:39:26Z")

</div>

I'm not sure exactly how `break_on_match` affects the addition of the `_grokparsefailure` tag, but if the tag is added unless _all_ expressions match then that's clearly the reason since /nexus doesn't match /nexus/content/repositories.

---

<div class="post-metadata">

### Author: ![simonrisberg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simonrisberg/32/3513_2.png) [@simonrisberg](https://discuss.elastic.co/u/simonrisberg)
#### Post date: [July 30, 2015, 2:45pm UTC](https://discuss.elastic.co/t/regular-expression-problem/26484/9 "2015-07-30T14:45:37Z")

</div>

I understand. Well I need to have the break\_on\_match function so I guess I'll just have to live with it.

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [July 30, 2015, 3:55pm UTC](https://discuss.elastic.co/t/regular-expression-problem/26484/10 "2015-07-30T15:55:36Z")

</div>

No, you don't need `break_on_match`. You could easily merge all three expressions into a single expression. Or, as mentioned previously, use a generic pattern to do the bulk of the parsing instead of reinventing the wheel.

---

<div class="post-metadata">

### Author: ![simonrisberg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simonrisberg/32/3513_2.png) [@simonrisberg](https://discuss.elastic.co/u/simonrisberg)
#### Post date: [July 31, 2015, 9:33am UTC](https://discuss.elastic.co/t/regular-expression-problem/26484/11 "2015-07-31T09:33:48Z")

</div>

I can see how I might be able to use a generic pattern on the the pattern "repositories" that I have created but I don't really see it happening on the "mytimestamp" part. I'm not entirely sure how to merged them into a single expression either. Wouldn't that look pretty strange?

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [July 31, 2015, 6:07pm UTC](https://discuss.elastic.co/t/regular-expression-problem/26484/12 "2015-07-31T18:07:21Z")

</div>

No, why would it be strange? But yes, becuase you're extracting the `repositories` field from the URI you can't use the predefined grok patterns out of the box but you could certainly use them as a starting point. You're attempting to parse a single line so it makes perfect sense to use a single expression for the parsing.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 5:33am UTC](https://discuss.elastic.co/t/regular-expression-problem/26484/13 "2017-07-06T05:33:11Z")

</div>


