Logstash event- Failed to Parse field

Elastic Stack Logstash
1

Hi.

I have this error in my logstash:

[2020-04-28T14:58:13,744][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"nonfood_pro-2020.04.26", :_type=>"logs", :routing=>nil}, #<LogStash::Event:0x3f9c10ba>], :response=>{"index"=>{"_index"=>"nonfood_pro-2020.04.26", "_type"=>"logs", "_id"=>"N4jdwHEB94vKD11jxvSI", "status"=>400, "error"=>{"type"=>"mapper_parsing
exception", "reason"=>"failed to parse field [MessageParam2] of type [float] in document with id 'N4jdwHEB94vKD11jxvSI'", "caused_by"=>{"type"=>"number_format_exception", "reason"=>"For input string: \"XAException.XAER_RMFAIL\""}}}}}
[2020-04-28T14:58:13,744][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"nonfood_pro-2020.04.26", :_type=>"logs", :routing=>nil}, #<LogStash::Event:0x29fd7147>], :response=>{"index"=>{"_index"=>"nonfood_pro-2020.04.26", "_type"=>"logs", "_id"=>"OIjdwHEB94vKD11jxvSI", "status"=>400, "error"=>{"type"=>"mapper_parsing
exception", "reason"=>"failed to parse field [MessageParam2] of type [float] in document with id 'OIjdwHEB94vKD11jxvSI'", "caused_by"=>{"type"=>"number_format_exception", "reason"=>"For input string: \"XAException.XAER_RMFAIL\""}}}}}

These are my filters:

	                grok {
	                    match => ["message", "(?m)(?<timestamp>%{YEAR}-%{MONTHNUM}-%{MONTHDAY}[T ]%{HOUR}:?%{MINUTE}(?::?%{SECOND})) +%{LOGLEVEL:logLevel} +\[%{GREEDYDATA:componentName}\] +\[%{GREEDYDATA:threadName}\:ipaddr=%{GREEDYDATA:iplist}\;path=%{GREEDYDATA:request}\;sessionid=%{NOTSPACE:jsessionid}\] +\[(userID: +%{NOTSPACE:profileid})?\] +%{GREEDYDATA:logMsg}"]
	                    match => ["message", "(?m)(?<timestamp>%{YEAR}-%{MONTHNUM}-%{MONTHDAY}[T ]%{HOUR}:?%{MINUTE}(?::?%{SECOND})) +%{LOGLEVEL:logLevel} +\[%{DATA}\] +\[%{DATA}\] +\[(userID: +%{NOTSPACE:profileid})?\] +%{GREEDYDATA} +\(thread=%{GREEDYDATA:threadName}\:ipaddr=%{GREEDYDATA:iplist}\;path=%{GREEDYDATA:request}\;sessionid=%{NOTSPACE:jsessionid}\, member=%{DATA}\)\: +%{GREEDYDATA:logMsg}"]
	                    overwrite => ["message"]
	                }
	                mutate {
	                    split => {
	                        "iplist" => ","
	                    }
	                    add_field => {
	                        "clientip" => "%{[iplist][0]}"
	                    }
	                }

Can anybody tell me what is going on or what I need to check?. I am lost.

2

elasticsearch is expecting the [MessageParam2] field to be a float. However, you have an event where the value of the field is "XAException.XAER_RMFAIL", which ES cannot parse as a float.

There could be an index template that tells ES that that field should be a float. Or you may need to create one that tells it that it is not (if ES just learned it by parsing the first document it indexed)

1 Like
3

@Badger Thank you very much.

One question. Can you tell me where this parameter: MessageParam2 comes from?. I cannot see any configuration file that tells logstash to split all the parameters into: MessageParam1, MessageParam2, ... I only have the filters above mentioned. Perhaps I am missing something?.

4

I have no idea.

5

@Badger But this parameter is not in the log file, so it has be added by logstash, am I right?

6

I do not know.

1 Like
7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.