Grok match error


(erkan) #1

hi,
./logstash -f /etc/logstash/conf.d/logstash.conf

example log:
[2018-09-26T10:04:12,150][INFO ][o.e.e.NodeEnvironment ] [sezen] using [1] data paths, mounts [(rootfs)]], net usable_space [43.8gb], net total_space [49.9gb], types [rootfs]

-------------------------logstash.conf-----------------------------------
input {
beats {
port => 5044
}
}
filter {
grok { match => { "message" => "[%{TIMESTAMP_ISO8601:timestamp}][%{DATA:loglevel}%{SPACE}][%{DATA:node}\s]\s[%{DATA:client}]\ %{GREEDYDATA:message}" } }
}
output {
elasticsearch {
hosts => ["192.168.1.10:9200"]
}
}

[ERROR] 2018-10-03 11:22:19.546 [[main]-pipeline-manager] pipeline - Error registering plugin {:pipeline_id=>"main", :plugin=>"#<LogStash::FilterDelegator:0x3a0f5986 @metric_events_out=org.jruby.proxy.org.logstash.instrument.metrics.counter.LongCounter$Proxy2 - name: out value:0, @metric_events_in=org.jruby.proxy.org.logstash.instrument.metrics.counter.LongCounter$Proxy2 - name: in value:0, @metric_events_time=org.jruby.proxy.org.logstash.instrument.metrics.counter.LongCounter$Proxy2 - name: duration_in_millis value:0, @id="e6ac64e9b8dff0409a3d1bf20f6ad4a18eead5ee767d4ae0c04865bd71e15939", @klass=LogStash::Filters::Grok, @metric_events=#LogStash::Instrument::NamespacedMetric:0x5d25af87, @filter=<LogStash::Filters::Grok match=>{"message"=>"\\[%{TIMESTAMP_ISO8601:timestamp}\\]\\[%{DATA:loglevel}%{SPACE}\\]\\[%{DATA:node}\\s\\]\\s\\[%{DATA:client}]\\ %{GREEDYDATA:message}"}, id=>"e6ac64e9b8dff0409a3d1bf20f6ad4a18eead5ee767d4ae0c04865bd71e15939", enable_metric=>true, periodic_flush=>false, patterns_files_glob=>"", break_on_match=>true, named_captures_only=>true, keep_empty_captures=>false, tag_on_failure=>["_grokparsefailure"], timeout_millis=>30000, tag_on_timeout=>"_groktimeout">>", :error=>"end pattern with unmatched parenthesis: /\[(?<TIMESTAMP_ISO8601:timestamp>(?:(?>\d\d){1,2})-(?:(?:0?[1-9]|1[0-2]))-(?:(?:(?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9]))[T ](?:(?:2[0123]|[01]?[0-9]):?(?:(?:[0-5][0-9]))(?::?(?:(?:(?:[0-5]?[0-9]|60)(?:[:.,][0-9]+)?)))?(?:(?:Z|+-)?)\]\[(?<DATA:loglevel>.?)(?:\s*)\]\[(?<DATA:node>.?)\s\]\s\[(?<DATA:client>.?)]\ (?GREEDYDATA:message.)/m", :thread=>"#<Thread:0x159bfda6 run>"}
[ERROR] 2018-10-03 11:22:19.549 [[main]-pipeline-manager] pipeline - Pipeline aborted due to error {:pipeline_id=>"main", :exception=>#<RegexpError: end pattern with unmatched parenthesis: /[(?<TIMESTAMP_ISO8601:timestamp>(?:(?>\d\d){1,2})-(?:(?:0?[1-9]|1[0-2]))-(?:(?:(?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9]))[T ](?:(?:2[0123]|[01]?[0-9]):?(?:(?:[0-5][0-9]))(?::?(?:(?:(?:[0-5]?[0-9]|60)(?:[:.,][0-9]+)?)))?(?:(?:Z|+-)?)][(?<DATA:loglevel>.
?)(?:\s*)][(?<DATA:node>.?)\s]\s[(?<DATA:client>.?)]\ (?GREEDYDATA:message.*)/m>, :backtrace=>["org/jruby/RubyRegexp.java:928:in initialize'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/jls-grok-0.11.5/lib/grok-pure.rb:127:incompile'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-grok-4.0.3/lib/logstash/filters/grok.rb:281:in block in register'", "org/jruby/RubyArray.java:1734:ineach'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-grok-4.0.3/lib/logstash/filters/grok.rb:275:in block in register'", "org/jruby/RubyHash.java:1343:ineach'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-grok-4.0.3/lib/logstash/filters/grok.rb:270:in register'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:242:inregister_plugin'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:253:in block in register_plugins'", "org/jruby/RubyArray.java:1734:ineach'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:253:in register_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:595:inmaybe_setup_out_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:263:in start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:200:inrun'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:160:in `block in start'"], :thread=>"#<Thread:0x159bfda6 run>"}
[ERROR] 2018-10-03 11:22:19.569 [Converge PipelineAction::Create] agent - Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create, action_result: false", :backtrace=>nil}
[INFO ] 2018-10-03 11:22:20.164 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9600}

ty for help


(Bardie) #2

Works with this:

\[%{TIMESTAMP_ISO8601:timestamp}\]\[%{DATA:loglevel}%{SPACE}\]\[%{DATA:node}\s\]\s\[%{DATA:client}\]\ %{GREEDYDATA:message}

put a back slash (\) before ([) (])the braces


(erkan) #3

Thank you so much.


(system) #4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.