Unable to pass the custom pattern

Elastic Stack Logstash
1

Hi Team,
I am trying to parse the time which is anchored

Eg <Jul 30, 2019 9:02:53 PM PDT>

I have created custom pattern it in different location
patterns_dir => [ "/etc/logstash/custompaterns/patterns" ]

WEBLOG_TIME %{MONTH} %{MONTHDAY}, %{YEAR} %{TIME} %{DL}

DL ([P|A]M]?)

But i try to execute the logstash, it error registering the pipeline repeatedly.

 Aug 20 06:50:21 RCOVLNX3403.corp.frk.com logstash[38953]: [2019-08-20T06:50:21,843][ERROR][logstash.pipeline        ] Error registering plugin {:pipeline_id=>"usde-intl-pipeline", :plugin=>"#<LogStash::FilterDelegator:0x499f6d9b @metric_events_out=org.jruby.proxy.org.logstash.instrument.metrics.counter.LongCounter$Proxy2 -  name: out value:0, @metric_events_in=org.jruby.proxy.org.logstash.instrument.metrics.counter.LongCounter$Proxy2 -  name: in value:0, @metric_events_time=org.jruby.proxy.org.logstash.instrument.metrics.counter.LongCounter$Proxy2 -  name: duration_in_millis value:0, @id=\"ca3c39d8a7e2324b52d76a242e13e248d36d53237d719e48c4d71bf5c65ac01f\", @klass=LogStash::Filters::Grok, @metric_events=#<LogStash::Instrument::NamespacedMetric:0x244d8dae>, @filter=<LogStash::Filters::Grok match=>{\"message\"=>\"####\\\\<%{WEBLOG_TIME:attimestamp}\\\\s%{DATA:ign}\\\\>\\\\s\\\\<%{LOGLEVEL:severity}\\\\>\\\\s\\\\<%{DATA:wls_topic}\\\\>\\\\s<%{DATA:hostname}\\\\>\\\\s\\\\<%{DATA:Wls_appname}\\\\>\\\\s\\\\<%{DATA:threadinfo}\\\\>\\\\s\\\\<\\\\<%{DATA:wls_kernel}\\\\>\\\\>\\\\s<%{DATA:empty1}\\\\>\\\\s\\\\<%{DATA:empty2}\\\\>\\\\s\\\\<%{NUMBER:notreqd}\\\\>\\\\s\\\\<%{DATA:anotherbea}\\\\>\\\\s\\\\<%{GREEDYDATA:Logmessage}\\\\>\"}, id=>\"ca3c39d8a7e2324b52d76a242e13e248d36d53237d719e48c4d71bf5c65ac01f\", patterns_dir=>[\"/etc/logstash/custompaterns/patterns\"], overwrite=>[\"message\"], enable_metric=>true, periodic_flush=>false, patterns_files_glob=>\"*\", break_on_match=>true, named_captures_only=>true, keep_empty_captures=>false, tag_on_failure=>[\"_grokparsefailure\"], timeout_millis=>30000, tag_on_timeout=>\"_groktimeout\">>", :error=>"pattern %{WEBLOG_TIME:attimestamp} not defined", :thread=>"#<Thread:0x5f692597@/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:46 run>"}
    Aug 20 06:50:22 RCOVLNX3403.corp.frk.com logstash[38953]: [2019-08-20T06:50:22,214][ERROR][logstash.pipeline        ] Pipeline aborted due to error {:pipeline_id=>"usde-intl-pipeline", :exception=>#<Grok::PatternError: pattern %{WEBLOG_TIME:attimestamp} not defined>, :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/jls-grok-0.11.5/lib/grok-pure.rb:123:in `block in compile'", "org/jruby/RubyKernel.java:1292:in `loop'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/jls-grok-0.11.5/lib/grok-pure.rb:93:in `compile'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-grok-4.0.3/lib/logstash/filters/grok.rb:281:in `block in register'", "org/jruby/RubyArray.java:1734:in `each'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-grok-4.0.3/lib/logstash/filters/grok.rb:275:in `block in register'", "org/jruby/RubyHash.java:1343:in `each'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-grok-4.0.3/lib/logstash/filters/grok.rb:270:in `register'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:242:in `register_plugin'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:253:in `block in register_plugins'", "org/jruby/RubyArray.java:1734:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:253:in `register_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:595:in `maybe_setup_out_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:263:in `start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:200:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:160:in `block in start'"], :thread=>"#<Thread:0x5f692597@/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:46 run>"}
2

Well apparently it is not finding the files in the patterns_dir. Try defining them in your config

grok {
    pattern_definitions {
        "WEBLOG_TIME" => "%{MONTH} %{MONTHDAY}, %{YEAR} %{TIME} %{DL}"
        "DL" => "([P|A]M]?)"
    }
    ...
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.