Hi,
I can't wrap my head around why I don't get this translate filter to work.
I have a bunch of IoT logfiles (csv) that I want to import.
- One of the fields (KO-ID) does contain an internal ID of the old log engine, for example "3051"
- another field (GrpAdr) contains the ID I need to work with in the future. Because it is not available for all log entries, I want to substitute this GrpAdr for a range of old IDs (3050 to 3101). In other words: if KO-ID is in the range above, use the lookup table to overwrite whatever there is in GrpAdr (it should be empty anyway).
KO-ID and GrpAdr both populate the correct fields when I import the log files. However, I cannot get the translation to work (also tried a dictionary instead of lookup file). GrpAdr is always empty for the values in the range, but it has values when they are in the original log.
current config:
filter {
...
if "knximport" in [tags] {
csv {
columns => [ "eventtime", "Funktion", "Geschoss", "Gewerk", "KNX-Name", "KNX-Wert", "KNX-Wert-Float", "Name", "Raum", "GrpAdr", "KOID", "PA", "Typ" ]
separator => ","
}
translate {
source => "[KOID]"
target => "[GrpAdr]"
dictionary => {
"3050" => "7/0/0"
"3051" => "7/0/1"
"3052" => "7/0/2"
...
"654" => "11/2/3"
"669" => "11/2/4"
}
fallback => "%{GrpAdr}"
}
I tried the field name with and without brackets, and several other things. However, the result is always the same:
Here is a snippet from what I am importing:
2023-09-10T10:30:00.150Z,,,,Zirkulation,-29.9,,no match,,-,3056,,INTERN
2023-09-10T10:30:00.156Z,,,,Brauchwasser Solltemperatur,50,,no match,,-,3060,,INTERN
2023-09-10T10:30:00.160Z,,,,Betriebsart,3,,no match,,-,3064,,INTERN
2023-09-10T10:30:00.165Z,,,,Fusspunkt,35,,no match,,-,3068,,INTERN
2023-09-10T10:30:00.170Z,,,,Heizgrenze Absenken,10,,no match,,-,3072,,INTERN
2023-09-10T10:30:00.175Z,,,,Spreizung Heizkreis,20,,no match,,-,3076,,INTERN
What am I missing?