OpenLdap log parsing pipeline issue

Hello, I am currently trying to parse some openldap logs and need some help from the ElasticSearch gods.

attached is the error message

[2019-10-23T00:03:53,115][ERROR][logstash.javapipeline    ] Pipeline aborted due to error {:pipeline_id=>"openldap", :exception=>#<RegexpError: undefined group option: /(?:(?:(?anonymous))|(?:dn=\"(?<DATA:bind_dn>.*?)\")) (?:(?:method=(?<WORD:bind_method>\b\w+\b))|(?:mech=(?<WORD:bind_mech>\b\w+\b) ssf=(?<INT:bind_ssf>(?:[+-]?(?:[0-9]+)))))(?:\s*)$/m>, :backtrace=>["org/jruby/RubyRegexp.java:940:in `initialize'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/jls-grok-0.11.5/lib/grok-pure.rb:127:in `compile'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-grok-4.1.1/lib/logstash/filters/grok.rb:274:in `block in register'", "org/jruby/RubyArray.java:1800:in `each'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-grok-4.1.1/lib/logstash/filters/grok.rb:268:in `block in register'", "org/jruby/RubyHash.java:1417:in `each'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-grok-4.1.1/lib/logstash/filters/grok.rb:263:in `register'", "org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:56:in `register'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:195:in `block in register_plugins'", "org/jruby/RubyArray.java:1800:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:194:in `register_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:468:in `maybe_setup_out_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:207:in `start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:149:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:108:in `block in start'"], :thread=>"#<Thread:0x738b9921 run>"}
[2019-10-23T00:03:53,143][ERROR][logstash.agent           ] Failed to execute action {:id=>:openldap, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<openldap>, action_result: false", :backtrace=>nil}

current logstash.conf file

input {
  beats {
   port => 5044
  }
}


filter {
 grok {
        match => [ "message", "(?:(?:<= (?:b|m)db_%{DATA:index_error_filter_type}_candidates: \(%{WORD:index_error_attribute_name}\) not indexed)|(?:ppolicy_%{DATA:ppolicy_op}: %{DATA:ppolicy_data})|(?:connection_input: conn=%{INT:connection} deferring operation: %{DATA:deferring_op})|(?:connection_read\(%{INT:fd_number}\): no connection!)|(?:conn=%{INT:connection} (?:(?:fd=%{INT:fd_number} (?:(?:closed(?: \(connection lost\)|))|(?:ACCEPT from IP=%{IP:src_ip}\:%{INT:src_port} \(IP=%{IP:dst_ip}\:%{INT:dst_port}\))|(?:TLS established tls_ssf=%{INT:tls_ssf} ssf=%{INT:ssf})))|(?:op=%{INT:operation_number} (?:(?:(?:(?:SEARCH )|(?:))RESULT (?:tag=%{INT:tag}|oid=(?:%{DATA:oid}(?:))) err=%{INT:error_code}(?:(?: nentries=%{INT:nentries})|(?:)) text=(?:(?:%{DATA:error_text})|(?:)))|(?:%{WORD:operation_name}(?:(?: %{DATA:data})|(?:))))))))%{SPACE}$" ] remove_tag => "_grokparsefailure"
        add_tag => "openldap"
    }
    if [operation_name] == "BIND" {
      grok {
        match => [ "data", "(?:(?:(?anonymous))|(?:dn=\"%{DATA:bind_dn}\")) (?:(?:method=%{WORD:bind_method})|(?:mech=%{WORD:bind_mech} ssf=%{INT:bind_ssf}))%{SPACE}$" ]
        remove_field => [ "data" ]
      }
    }
    if [operation_name] == "SRCH" {
      grok {
        match => [ "data", "(?:(?:base=\"%{DATA:search_base}\" scope=%{INT:search_scope} deref=%{INT:search_deref} filter=\"%{DATA:search_filter}\")|(?:attr=%{DATA:search_attr}))%{SPACE}$" ]
        remove_field => [ "data" ]
      }
    }
    if [operation_name] == "MOD" {
      grok {
        match => [ "data", "(?:(?:dn=\"%{DATA:mod_dn}\")|(?:attr=%{DATA:mod_attr}))%{SPACE}$" ]
        remove_field => [ "data" ]
      }
    }
    if [operation_name] == "MODRDN" {
      grok {
        match => [ "data", "dn=\"%{DATA:modrdn_dn}\"%{SPACE}$" ]
        remove_field => [ "data" ]
      }
    }
    if [operation_name] == "ADD" {
      grok {
        match => [ "data", "dn=\"%{DATA:add_dn}\"%{SPACE}$" ]
        remove_field => [ "data" ]
      }
    }
    if [operation_name] == "DEL" {
      grok {
        match => [ "data", "dn=\"%{DATA:del_dn}\"%{SPACE}$" ]
        remove_field => [ "data" ]
      }
    }
    if [operation_name] == "CMP" {
      grok {
        match => [ "data", "dn=\"%{DATA:cmp_dn}\" attr=\"%{DATA:cmp_attr}\"%{SPACE}$" ]
        remove_field => [ "data" ]
      }
    }
    if [operation_name] == "EXT" {
      grok {
        match => [ "data", "oid=%{DATA:ext_oid}%{SPACE}$" ]
        remove_field => [ "data" ]
      }
    }
    if [ppolicy_op] == "bind" {
      grok {
        match => [ "ppolicy_data", "(?:(?:Entry %{DATA:ppolicy_bind_dn} has an expired password: %{INT:ppolicy_grace} grace logins)|(?:Setting warning for password expiry for %{DATA:ppolicy_bind_dn} = %{INT:ppolicy_expiration} seconds))%{SPACE}$" ]
        remove_field => [ "ppolicy_data" ]
      }
    }
  }

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.