# Grok fails on a valid regex

**URL:** <https://discuss.elastic.co/t/grok-fails-on-a-valid-regex/231781>\
**Category:** Logstash\
**Created:** [May 8, 2020, 6:40pm UTC](https://discuss.elastic.co/t/grok-fails-on-a-valid-regex/231781 "2020-05-08T18:40:01Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![konart](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/konart/32/67941_2.png) [@konart](https://discuss.elastic.co/u/konart)\
**Post date:** [May 8, 2020, 6:40pm UTC](https://discuss.elastic.co/t/grok-fails-on-a-valid-regex/231781/1 "2020-05-08T18:40:01Z")

</div>

Log:

```auto
    [logstash.javapipeline][main] Pipeline aborted due to error {:pipeline_id=>"main", :exception=>#<RegexpError: unmatched close parenthesis: /(?<loglevel>DBG|INF|WRN)(?(?=(?:.*(?:X-Request-Id|CorrelationId)))(?:.*)(?<=X-Request-Id=.|CorrelationId=.)(?<request_id>[^]*))/m>, :backtrace=>["org/jruby/RubyRegexp.java:942:in `initialize'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/jls-grok-0.11.5/lib/grok-pure.rb:127:in `compile'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-grok-4.2.0/lib/logs
     tash/filters/grok.rb:284:in `block in register'", "org/jruby/RubyArray.java:1814:in `each'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-grok-4.2.0/lib/logstash/filters/grok.rb:278:in `block in register'", "org/jruby/RubyHash.java:1428:in `each'", "/usr/sha
     re/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-grok-4.2.0/lib/logstash/filters/grok.rb:273:in `register'", "org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:56:in `register'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:200:in `block
      in register_plugins'", "org/jruby/RubyArray.java:1814:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:199:in `register_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:502:in `maybe_setup_out_plugins'", "/usr/share/logstash/
     logstash-core/lib/logstash/java_pipeline.rb:212:in `start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:154:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:109:in `block in start'"], "pipeline.sources"=>["/usr/share/logstas
     h/pipeline/logstash.conf"], :thread=>"#<Thread:0x2c530ae9 run>"}

```

Regex in question:

`(?<loglevel>DBG|INF|WRN)(?(?=(?:.*(?:X-Request-Id|CorrelationId)))(?:.*)(?<=X-Request-Id=.|CorrelationId=.)(?<request_id>[^]*))`

Regex is valid and can be tested in [https://regex101.com/](https://regex101.com/) fro example.

Test log to test on:

`10:58AM DBG gateway-auth -> gateway-bff eX-Request-Id=e2d071784d6a1468f2b91fcf4973e806f eExchange=egateway eReplyTo=e eRoutingKey=egateway-bff ehandler=e"RQM message published"`

Any suggestions?

---

<div class="post-metadata">

**Author:** ![andres-perez](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andres-perez/32/136461_2.png) [@andres-perez](https://discuss.elastic.co/u/andres-perez)\
**Post date:** [May 9, 2020, 4:50pm UTC](https://discuss.elastic.co/t/grok-fails-on-a-valid-regex/231781/3 "2020-05-09T16:50:57Z")

</div>

Hi!

> [@konart](#):
>
> Regex is valid and can be tested in [https://regex101.com/](https://regex101.com/) fro example.

well, talking about regex, a pattern can be valid... depending on the supported regex flavour 🤔

Your example seems a valid PCRE regex, but [grok uses JRuby and onigurima regex engine](https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html#_regular_expressions), so you should test first in other tools like [grok Debugger](https://grokdebug.herokuapp.com/) or [grok constructor](https://grokconstructor.appspot.com/do/match).

I found that neither of them show more detailed information than the logstash output you got, so another good try is "rubular". Your original regex throws _"Invalid conditional pattern"_ .

Adding a colon to the conditional block seems to work: `(?:(?=` instead of `(?(?=`  
Check it by yourself: [Rubular: (?\<loglevel\>DBG|INF|WRN)(?:(?=(?:.\*(?:X-Request-Id|CorrelationId)))(?:.\*)(?\<=X-Request-Id=.|CorrelationId=.)(?\<request\_id\>[^]\*))](https://rubular.com/r/PlpUNuIGfp387w)

* * *

Having said that, It looks a bit overcomplicated expression (conditionals and lookbehind...) just to extract a couple of strings, I would advise trying a simpler approach from scratch.

It should also gain a lot of efficiency by adding an anchor to the begining of the line, otherwise it will iterate a bazillion times from the begining to the end of the log message looking for the terms contained in your pattern.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 6, 2020, 4:51pm UTC](https://discuss.elastic.co/t/grok-fails-on-a-valid-regex/231781/4 "2020-06-06T16:51:08Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.