Iterate_on in Logstash no working properly

I don't know if I'm using this wrongly or if it's a problem with the filter, but, I have the following structure:
{
...,
"eventDataSplit" => {
"1" => {
"valueKey" : "1.1.1"
},
"2" => {
"valueKey" : "2.1.1"
},
"3" => {
"valueKey" : "3.1.1"
}
}

And I want to translate the "valueKey" from a .csv file or .json using the property iterate_on:

CSV (tried with quotes too)
1.1.1,one thing
2.1.1,another thing

JSON
{
"1.1.1":"one thing",
"2.1.1":"another thing"
}

Filter

translate {
iterate_on => "[eventDataSplit]"
dictionary_path => "\path-to\translateKey.json"
field => "[valueKey]"
destination => "[valueDescription]"
fallback => "-1"
}

The expected result is:
{
...,
"eventDataSplit" => {
"1" => {
"valueKey" : "1.1.1",
"valueDescription": "one thing"
},
"2" => {
"valueKey" : "2.1.1",
"valueDescription": "another thing"
},
"5" => {
"valueKey" : "5.1.1",
"valueDescription": "-1"
}
}

But something really strange happen and iterate_on "jumps" one of the values and I get nothing, not even the fallback value:

{
...,
"eventDataSplit" => {
"1" => {
"valueKey" : "1.1.1",
"valueDescription": "one thing"
},
"2" => {
"valueKey" : "2.1.1",
"valueDescription": "another thing"
},
"5" => {
"valueKey" : "5.1.1"
}

}

I don't know if all this json and csv thing was needed, but I really need some help. Am I doing something wrong?
Currently using logstash 7.8.1
Thanks

The way I read the documentation, the fallback value will only be set of no matches are found for the full event and not per translation.

I think I didn't understand exactly what you said, but I expected the filter to work as show in the documentation example for "iterate_on" case 2.
Here's one of my output:

"eventDataSplit" => {
"2" => {
"valueKey" => "774.2.-8.076126",
"valueDescription" => "-1",
"value" => "-8.076126"
},
"6" => {
"valueKey" => "774.6.210",
"valueDescription" => "-1",
"value" => "210"
},
"7" => {
"valueKey" => "774.7.0",
"value" => "0"
},
"1" => {
"valueKey" => "774.1.1",
"valueDescription" => "Válida.",
"value" => "1"
},
"3" => {
"valueKey" => "774.3.-48.229557",
"valueDescription" => "-1",
"value" => "-48.229557"
},
"4" => {
"valueKey" => "774.4.3244",
"valueDescription" => "-1",
"value" => "3244"
},
"5" => {
"valueKey" => "774.5.451",
"valueDescription" => "-1",
"value" => "451"
}
}

Of all of valueKey value, only the "774.1.1" was present in the dictionary file.
In the next example, there's only one eventDataSplit value and the valueKey value is present in the dictionary file:

"eventDataSplit" => {
"1" => {
"valueKey" => "4097.1.1",
"value" => "1"
}
}

It's like the "iterate_on" jumped those values and it's always the same.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.