# Pattern not defined even though the first matches

**URL:** <https://discuss.elastic.co/t/pattern-not-defined-even-though-the-first-matches/94929>\
**Category:** Logstash\
**Created:** [July 28, 2017, 9:58am UTC](https://discuss.elastic.co/t/pattern-not-defined-even-though-the-first-matches/94929 "2017-07-28T09:58:13Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Hisushi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hisushi/32/20551_2.png) [@Hisushi](https://discuss.elastic.co/u/Hisushi)\
**Post date:** [July 28, 2017, 9:58am UTC](https://discuss.elastic.co/t/pattern-not-defined-even-though-the-first-matches/94929/1 "2017-07-28T09:58:13Z")

</div>

Hi everybody,

I'm pretty new to the ELK stack but I think I understand the basics.

So far, I've been able to define a pattern to match my log file and an additional grok pattern. I also checked if it works as expected by using the [Grok Constructor](http://grokconstructor.appspot.com/do/match).

My grok patterns are defined as follows

```
CLIENT \d{1,3}
TEXT (?:(?!\s{2,}).)*

```

whereas my configuration looks like this:

```
input {
  file {
    path => ["/media/SAP/logs"]
    type => ["security_audit_log"]
    start_position => ["beginning"]
    codec => plain {
      charset => "CP1252"	
    }
  }
}

filter {
  if [type] == "security_audit_log" {
    grok {
      patterns_dir => ["/media/ELK/logstash-5.4.1/config/patterns"]
      match => { 
        "message" => "\|%{DATE_EU:date}?%{SPACE}?\|%{TIME:time}?%{SPACE}?\|%{CLIENT:client}?%{SPACE}?\|%{USERNAME:user}?%{SPACE}?\|%{TEXT:terminal}?%{SPACE}?\|%{TEXT:tcode}?%{SPACE}?\|%{TEXT:program}?%{SPACE}?\|%{TEXT:text}?%{SPACE}?\|"
      }
    }
    mutate {
      add_field => {
        "timestamp" => "%{date} %{time}" 
      }
      remove_field => ["date", "time"]
    }
    date {
      match => ["timestamp" , "dd.MM.yyyy HH:mm:ss"]
      timezone => ["Europe/Berlin"]
      target => ["@timestamp"]
    }
  }
}

output {
  if [type] == "security_audit_log" {
    elasticsearch {
      hosts => ["localhost:9200"]
      index => ["logstash-security_audit_log"]
    }
    stdout {
      codec => rubydebug
    }
  }	
}

```

Now, when runnin ./bin/logstash as I've downloaded the zip, I receive the following error:

```
elk@ELK:/media/ELK/logstash-5.4.1/bin$ ./logstash
Sending Logstash's logs to /media/ELK/logstash-5.4.1/logs which is now configured via log4j2.properties
[2017-07-28T11:09:57,742][WARN][logstash.filters.date] Date filter now use BCP47 format for locale, replacing underscore with dash
[2017-07-28T11:09:58,398][INFO][...]
[2017-07-28T11:09:58,400][INFO][...]
[2017-07-28T11:09:58,576][WARN][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>#<URI::HTTP:0x6d065f34 URL:http://localhost:9200/>}
[2017-07-28T11:09:58,578][INFO][...]
[2017-07-28T11:09:58,657][INFO][...]
[2017-07-28T11:09:58,670][INFO][...]
[2017-07-28T11:09:58,768][ERROR][logstash.pipeline] Error registering plugin {:plugin=>"#<LogStash::FilterDelegator:0x5aa1742e @id=\"36038c963454ac945ea4e7fd04113944e40ae1c9-3\", @klass=LogStash::Filters::Grok, @metric_events=#<LogStash::Instrument::NamespacedMetric:0x5b4d7979 @metric=#<LogStash::Instrument::Metric:0x78a26672 @collector=#<LogStash::Instrument::Collector:0x48f5595f @agent=nil, @metric_store=#<LogStash::Instrument::MetricStore:0x1924546b @store=#<Concurrent::Map:0x29127087 @default_proc=nil>, @structured_lookup_mutex=#<Mutex:0x42e81221>, @fast_lookup=#<Concurrent::Map:0x20358482 @default_proc=nil>>>>, @namespace_name=[:stats, :pipelines, :main, :plugins, :filters, :\"36038c963454ac945ea4e7fd04113944e40ae1c9-3\", :events]>, @logger=#<LogStash::Logging::Logger:0x2125c64d @logger=#<Java::OrgApacheLoggingLog4jCore::Logger:0x6831045c>>, @filter=<LogStash::Filters::Grok patterns_dir=>[\"/media/ELK/logstash-5.4.1/config/patterns\"], match=>{\"message\"=>\"\\\\|%{DATE_EU:date}?%{SPACE}?\\\\|%{TIME:time}?%{SPACE}?\\\\|%{CLIENT:client}?%{SPACE}?\\\\|%{USERNAME:user}?%{SPACE}?\\\\|%{TEXT:terminal}?%{SPACE}?\\\\|%{TEXT:tcode}?%{SPACE}?\\\\|%{TEXT:program}?%{SPACE}?\\\\|%{TEXT:text}?%{SPACE}?\\\\|\"}, id=>\"36038c963454ac945ea4e7fd04113944e40ae1c9-3\", enable_metric=>true, periodic_flush=>false, patterns_files_glob=>\"*\", break_on_match=>true, named_captures_only=>true, keep_empty_captures=>false, tag_on_failure=>[\"_grokparsefailure\"], timeout_millis=>30000, tag_on_timeout=>\"_groktimeout\">>", :error=>"pattern %{TEXT:terminal} not defined"}
[2017-07-28T11:09:58,795][ERROR][logstash.agent] Pipeline aborted due to error {:exception=>#<Grok::PatternError: pattern %{TEXT:terminal} not defined>, :backtrace=>["/media/ELK/logstash-5.4.1/vendor/bundle/jruby/1.9/gems/jls-grok-0.11.4/lib/grok-pure.rb:123:in `compile'", "org/jruby/RubyKernel.java:1479:in `loop'", "/media/ELK/logstash-5.4.1/vendor/bundle/jruby/1.9/gems/jls-grok-0.11.4/lib/grok-pure.rb:93:in `compile'", "/media/ELK/logstash-5.4.1/vendor/bundle/jruby/1.9/gems/logstash-filter-grok-3.4.0/lib/logstash/filters/grok.rb:286:in `register'", "org/jruby/RubyArray.java:1613:in `each'", "/media/ELK/logstash-5.4.1/vendor/bundle/jruby/1.9/gems/logstash-filter-grok-3.4.0/lib/logstash/filters/grok.rb:280:in `register'", "org/jruby/RubyHash.java:1342:in `each'", "/media/ELK/logstash-5.4.1/vendor/bundle/jruby/1.9/gems/logstash-filter-grok-3.4.0/lib/logstash/filters/grok.rb:275:in `register'", "/media/ELK/logstash-5.4.1/logstash-core/lib/logstash/pipeline.rb:268:in `register_plugin'", "/media/ELK/logstash-5.4.1/logstash-core/lib/logstash/pipeline.rb:279:in `register_plugins'", "org/jruby/RubyArray.java:1613:in `each'", "/media/ELK/logstash-5.4.1/logstash-core/lib/logstash/pipeline.rb:279:in `register_plugins'", "/media/ELK/logstash-5.4.1/logstash-core/lib/logstash/pipeline.rb:289:in `start_workers'", "/media/ELK/logstash-5.4.1/logstash-core/lib/logstash/pipeline.rb:214:in `run'", "/media/ELK/logstash-5.4.1/logstash-core/lib/logstash/agent.rb:398:in `start_pipeline'"]}
[2017-07-28T11:09:58,859][INFO][...]
[2017-07-28T11:10:01,830][WARN][...]

```

whereas the most important part seems to be the **pattern %{TEXT:terminal} not defined** part. I'm wondering: why is that? As the _CLIENT_ patterns seems to be applied, why does logstash not find the second one?

I've already been looking for a solution here but the only hints that I've found is checking the ownership of the pattern file/directory (which applies) and using the complete path to the patterns directory (what I did as well).

Any suggestions why this error is displayed? Or even a solution?

Best regards and thanks in advance,

Hisushi

**Edit**  
As suggested in another post, I used logstash with --debug which shows me the following:

```
[2017-07-31T10:18:57,996][DEBUG][logstash.filters.grok] Adding pattern {"CLIENT"=>"(\\d{1,3})\r\rTEXT ((?:(?!\\s{2,}).)*)"}

```

Why is that?

---

<div class="post-metadata">

**Author:** ![Hisushi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hisushi/32/20551_2.png) [@Hisushi](https://discuss.elastic.co/u/Hisushi)\
**Post date:** [August 7, 2017, 7:32am UTC](https://discuss.elastic.co/t/pattern-not-defined-even-though-the-first-matches/94929/2 "2017-08-07T07:32:03Z")

</div>

Anybody?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 4, 2017, 7:32am UTC](https://discuss.elastic.co/t/pattern-not-defined-even-though-the-first-matches/94929/3 "2017-09-04T07:32:03Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.