Translate multiple fields in an event

Elastic Stack Logstash
1

Hi,

I have events going into logstash. Each event has a number of fields two of which I would like to translate
The fields are customer_id and integrator_id.
I would like to add fields customer_name and integrator_name, the "id-to-name" translations coming from csv files.
I can translate one field but how would two be translated.

logstash.conf has: (obviously this doesn't work)
...
translate {
dictionary_path => "/etc/logstash/clients.csv"
field => "[client_id]"
add_field => { "client_name" => "%{[translation]}" }
}
translate {
dictionary_path => "/etc/logstash/integrators.csv"
field => "[integrator_id]"
add_field => { "integrator_name" => "%{[translation]}" }
}

...

Thanks in advance

2

There is no need to add a field if you instead specify the destination of the translation in the translate filter.

3

Christian,

Thankyou for your reply.
In my case, client_id is numeric and client_name text. Also client_name does not exist and I need it for later.

I also have the integrator_id field which I would like to translate at the same time as the client_id but am not sure how to specify two different fields for translation in the same block of code.

4

Per Translate filter plugin | Logstash Reference [8.11] | Elastic, value type is string so it takes only one field per translate block. Also, it is stated clearly in the docs that

If this field is an array, only the first value will be used.

This filter block should work for you

translate {
    dictionary_path => "/etc/logstash/clients.csv"
    field => "client_id"
    destination => "client_name"
}

translate {
    dictionary_path => "/etc/logstash/integrators.csv"
    field => "integrator_id"
    destination => "integrator_name"    
}

Do you mean LS fails to translate one of the fields in two separate translate blocks? If it fails when you try to put two fields into the field => setting, the reason is above.

In addition, numeric keys must be double quoted Translate filter plugin | Logstash Reference [8.11] | Elastic

5

I agree with tsangdl. Sounds like all your data is in a single event so the below should work. If the events were not of same format, or you are processing different pieces, you could use IF statements. This should work:
filter {
translate {
dictionary_path => "/etc/logstash/clients.csv"
field => "client_id"
destination => "client_name"
}
translate {
dictionary_path => "/etc/logstash/integrators.csv"
field => "integrator_id"
destination => "integrator_name"
}
}

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.