Logstash Key Value Filter unable to parse specific text

Elastic Stack Logstash
#1

I am trying to use Logstash's Key Value (kv) filter to parse logs. It specifically fails for a pair where key is "valid". see below example:

Log message: "versionInfo=VersionInfo{valid=test}"

Logstash filter config snippet for Key value pair:

filter {
kv {
source => "message"
field_split => "\{,\}"
trim => "\{\}\[\]"
}
}

Error:

"_type"=>"log", "_id"=>"AVcOLhG06gVwm51zjV4A", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [valid]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Invalid format: "test""}}}}, :level=>:warn}

But the same filter parses the message successfully if i remove the "valid" text and replace it with some other text like:

"versionInfo=VersionInfo{field=test}"

What is special for the "valid" text because of which parsing fails?

#2

This isn't the kv filter complaining, it's Elasticsearch. What's the mapping of the valid field?

#3

Hi Thanks for your reply. For now i've resolved it by using mutate filter and replacing valid text with _valid.
But for my knowledge and reference I would like to understand what do you mean by mapping? The log message has the word "valid" as is in it.
Excerpts from the log:

"versionInfo=VersionInfo{valid=test}"

#4

For now i've resolved it by using mutate filter and replacing valid text with _valid.

You should treat all field names that begin with underscore as reserved, i.e. don't use them.

But for my knowledge and reference I would like to understand what do you mean by mapping?

Please read about mappings in the Elasticsearch documentation. The valid field in your index has most likely been mapped as something that isn't compatible with the string "test".

#5