Regex \d or \w not working any longer in 7.1.1, but in 6.6.2: unmatched range specifier in char-class:

[asp]

Hi,

I am getting following filter in LS 7.1.1:

[2019-06-18T09:31:39,304][ERROR][logstash.javapipeline    ] Pipeline aborted due to error {:pipeline_id=>"auskunft_json", :exception=>#<RegexpError: unmatched range specifier in char-class: /^\[(?<FAST_TIMESTAMP:logTime>[\d-: ,]{23})\] (?<GREEDYDATA:myMessage>.*)/m>, :backtrace=>["org/jruby/RubyRegexp.java:940:in `initialize'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/jls-grok-0.11.5/lib/grok-pure.rb:127:in `compile'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-grok-4.0.4/lib/logstash/filters/grok.rb:281:in `block in register'", "org/jruby/RubyArray.java:1792:in `each'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-grok-4.0.4/lib/logstash/filters/grok.rb:275:in `block in register'", "org/jruby/RubyHash.java:1419:in `each'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-grok-4.0.4/lib/logstash/filters/grok.rb:270:in `register'", "org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:56:in `register'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:191:in `block in register_plugins'", "org/jruby/RubyArray.java:1792:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:190:in `register_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:446:in `maybe_setup_out_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:203:in `start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:145:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:104:in `block in start'"], :thread=>"#<Thread:0x16cbfc4c run>"}
[2019-06-18T09:31:39,319][ERROR][logstash.agent           ] Failed to execute action {:id=>:auskunft_json, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<auskunft_json>, action_result: false", :backtrace=>nil}

The filter is running without this issue in LS 6.6.2.

Here is my filter which causes the trouble:

grok
    {
            match => ['message','^\[%{FAST_TIMESTAMP:logTime}\] %{GREEDYDATA:myMessage}']
            patterns_dir => ['${GLOBAL_GROK_PATTERN_DIR}']
    }

I hunted it down to the regex behind FAST_TIMESTAMP:

# causing issue
FAST_TIMESTAMP [\d-: ,]{23}

# causing issue
FAST_TIMESTAMP [\w-: ,]{23}

# fixing issue
FAST_TIMESTAMP [[0-9]-: ,]{23}

Seems as if regex "symbols" for word, number, etc. are not working any longer. I didn't find a corresponding breaking change in the documentation.

I haven't tried if "symbols" like \s, \S, etc. are still working.
Do I really have to alter all my grok parsings where such symbols are used, or is there a way to enable them again?

Thanks, Andreas

[Badger]

@asp

[\d-: ,]{23}

In a character group (i.e. inside [ and ]) a - expresses a range. So that will match everything in the range of characters from \d to :, space or comma. I suspect you mean to use

[\d\-: ,]{23}

[asp]

@Badger

[\d-: ,]{23}

Great, thanks for your reply. That's it. Seems that new logstash is enforcing the rules more than older versions, but after your hint it works.