So, this is my output configuration:
output
{
if [field] == "ABC" and [OK] == "Yes"
{
elasticsearch
{
hosts => "localhost:9200"
user => "xxx"
password => "xxx"
index => "abc"
}
}
else if [field] == "DEF" and [OK] == "Yes"
{
elasticsearch
{
hosts => "localhost:9200"
user => "xxx"
password => "xxx"
index => "def"
}
}
else
{
elasticsearch
{
hosts => "localhost:9200"
user => "xxx"
password => "xxx"
index => "other"
}
}
stdout
{
codec => rubydebug
}
}
Everything work good, if field exist it's throwing my logs to properly indices.
But the error show up, when the field has another value than "abc" or "def".
Output config say us that, for example: log with field with value "xyz" should go to index "other",
but log didn't appear there, and in logstash stdout i can find error:
"error"=>{"type"=>"mapper_parsing_exception",
"reason"=>"failed to parse [field]", "caused_by"=>{"type"=>"number_format_exception",
"reason"=>"For input string: "xyz""}}}}, :level=>:warn}
I'm pretty sure that, this error is related to output config.
If statement didn't work properly?
It looks like ES is complaining because the field named field is mapped as an integer or float but you're trying to add a document containing the string "xyz".
if [field] != "DEF" and [field] != "ABC"
{
.
.
.
index => "other"
}
also didn't work.
When i put an integer to [field], it's asking for another field which is not integer. This is weird...
Any ideas how to solve that?
also didn't work.
Please show the error message.
When i put an integer to [field], it's asking for another field which is not integer. This is weird...
Why? It checks the fields in order, and when you've fixed the first non-integer field it'll look at the next one and complain about that.
Yes, it's normal, I know, but why it asks for Integer? If I put a proper name to [field] - it's adding whole log to correct index, if i put any other name there, it's showing me an error, but it should add that log to index with not properly written names (index => "other")... This is how IF ELSE should work, no?
What exactly is the problem here? That Logstash is attempting to send events to the wrong index or that it's not able to send events to the right index because ES returns mapper_parsing_exception?
Logstash is not able to send events to the right index because it returns mapper_parsing_exception.
Just check what i write in 1st post.
First IF statement is working good, if there is good value of field, the event go to right index.
Second ELSE IF statement also working well, events go to right index.
But last, ELSE is not working well... I thought, if field value is "blebleble" or anything, the event should go to the index named "other" but it returns mapper_parsing_exception...
Logstash is not able to send events to the right index because it returns mapper_parsing_exception.
Okay, so let's not derail the discussion by talking about if statements when those appear to be working just fine. Either stop trying to store strings in the integer field or map that field as a string. That requires the index to be reindexed.
1 Like