Shibboleth Auth Logs _grokparsefailure

[Dr_Natas]

How filebeat sends my shibboleth logs.
/etc/filebeat/filebeat.yml
Note: Code is truncated.

#--------------------------- Prospectors Path ---------------------------------
#filebeat.config.inputs:
#  enabled: true
#  path: /etc/filebeat/conf.d/*.yml
#
filebeat.config.prospectors:
enabled: true
path: /etc/filebeat/conf.d/*.yml

This is my filebeat config for shibboleth log shipping.
/etc/filebeat/conf.d/shibb.yml

====================================SHIBBOLETH========== =========================================
- type: log 
  paths:
  - /var/log/shibboleth/idp-process.log

exclude_lines: ['^DBG']
tags: ["shibboleth"]

# Include lines. A list of regular expressions to match. It exports the lines that are
# matching any regular expression from the list.
#include_lines: ['^ERR', '^WARN']

# Exclude files. A list of regular expressions to match. Filebeat drops the files that
# are matching any regular expression from the list. By default, no files are dropped.
exclude_files: ['.gz$']

# Optional additional fields. These fields can be freely picked
# to add additional information to the crawled log files for filtering
fields:
  level: debug
  review: 1

This is my shiv-dev code.

input {
generator {
	count => 1 
	message => "2018-09-27 21:23:54,633 - INFO [org.ldaptive.auth.Authenticator:282] - Authentication succeeded for dn: CN=jane.doe192,OU=Students,DC=three,DC=fifty,DC=onion"

}
}

filter {

grok { 
	match => { 
		"message" => "%{TIMESTAMP_ISO8601:logdate} - %{LOGLEVEL:log_level} (?<ldap-notes>\[[a-zA-Z0-9.:]*\]) - Authentication %{DATA:Success} for dn: %{GREEDYDATA:dubs}"
	}
}

kv {
	source => "dubs"
	field_split => ","
}

mutate {
   gsub => ["[dubs]", "DC|OU|CN|=", ""]
   split => { "[dubs]" => "," } # creates an array of strings in the [message] field]
   add_field => {
		"Common Name" => "%{[dubs][2]}.%{[dubs][3]}.%{[dubs][4]}"
   }
   remove_field => [ "DC" ]

   remove_tag => [ "othertags" ]
   }



mutate {
    # Original message has been fully parsed, so remove it.
    remove_field => [ "message" ]
	# Remove kv_pairs to avoid duplicate logs
    remove_field => [ "dubs" ]
}

}

output { stdout { codec => rubydebug } }

Sample output of dev code.

{
 "@timestamp" => 2018-10-05T18:49:18.663Z,
    "logdate" => "2018-09-27 21:23:54,633",
 "ldap-notes" => "[org.ldaptive.auth.Authenticator:282]",
   "@version" => "1",
    "Success" => "succeeded",
       "host" => "localhost",
   "sequence" => 0,
  "log_level" => "INFO",
         "CN" => "jane.doe192",
"Common Name" => "three.fifty.onion",
         "OU" => "Students"
}

I'm getting a _grokparsefailure and for the life of me can't figure out why I'm getting this error.

I was able to turn on debbuging.

[2018-10-05T14:39:23,556][DEBUG][logstash.pipeline        ] filter received {"event"=>{"beat"=>{"name"=>"logserver", "hostname"=>"local.host.name", "version"=>"6.3.2"}, "prospector"=>{"type"=>"log"}, "message"=>"2018-10-05 21:27:41,822 - INFO [org.ldaptive.auth.Authenticator:282] - Authentication failed for dn: CN=jane.doe057,OU=Staff,DC=three,DC=fifty,DC=onion", "tags"=>["shibboleth", "beats_input_codec_plain_applied"], "offset"=>27724776, "source"=>"/var/log/shibboleth/idp-process.log", "@timestamp"=>2018-10-05T21:39:21.163Z, "host"=>{"name"=>"serverSendingLogs"}, "fields"=>{"level"=>"debug", "env"=>"production", "review"=>1}, "input"=>{"type"=>"log"}, "@version"=>"1"}}