How to reference value of field in translate?

Elastic Stack Logstash
1

I would like to use the device name I have in the dictionary_path in the translate filter but I am having issues with it. is it even possible?

something like

translate {
field => "name"
destination => "lookUp"
dictionary_path => "/name.yml"
}

2

Hi,

yes it is possible I am using the translate feature a lot.

translate {
  field => "name"
  destination => "lookUp"
  dictionary_path => "name.yml"
}

the issue is with /name.yml you say to logstash to look at the root folder /. You just want to use name.yml if it is inside the same directory as your pipeline is.

3

but will name.yml be the value inside the field "name"?

4

https://www.elastic.co/guide/en/logstash/current/plugins-filters-translate.html No that is the full path to the file where you store your map.

Let's say you want to replace the short hostname of your machine with a full name.
shortname = 'philippMacBook'
fullname = 'philippkahrsFancyMacBookWithTouchBar'

translate{
  field => "shortname"
  destination => "fullname"
  dictionary_path => "names.yml"
}

Your names.yml would look like this

philippMacBook: "philippkahrsFancyMacBookWithTouchBar"

so in the names.yml you need to have the value of the field you are looking for stored as a key and the value you want to have in destination stored as a value.

I hope that helps

5

sorry I mean I want to use the value stored in the name field like a variable to look up the specific .yml file for the translation

6

I do not know to be honest. I think everything as a value to dictionary_path is treated as a string and does not resolve variables.

Could you explain what you are trying to achieve with the translate filter?

7

I am trying to look up a number associated with a name but many names have the same number so I would need multiple different .yml files to match the correct names and numbers.. if that makes sense

8

Ok so from my understanding you have a single key (name) that has multiple values (numbers)?
You would still need to put them all into a single yaml or json with dictionaries.

It is possible to provide multi-valued dictionary values. When using a YAML or JSON dictionary, you can have the value as a hash (map) or an array datatype.
https://www.elastic.co/guide/en/logstash/current/plugins-filters-translate.html#plugins-filters-translate-dictionary

Let's say you have the following key values.

philippsMacBook:
    - philipp
    - macbook
    - touchbar
    - cat

your translate should look like this

translate{
  field => "shortname"
  destination => "[fullnames]"
  dictionary_path => "names.yml"
}

since you are writing an array back. But you still need to combine all your yaml files into one.

9

i think i have not been very clear and that's my mistake let me try again

i have names of brands and each brand has multiple numbers associated with them ( part number) and each number is associated with a model but the problem is i have the same number used often for each of the brands.

so say i have fields brand and part_number and i was to translate the part number to the correct model based on the brand

translate {
field => "part_number"
destination => "model"
dictionary_path => %{brand}.yml
}

so say Apple was in the field brand it would use apple.yml in place of %{brand}.yml like a variable

again i do not know if this is possible i was hoping that i could use a variable

10

I understand your concept now. I do not think that this is possible, since the value of dictionary_path will possible be passed directly, you could open a github issue on that one here https://github.com/logstash-plugins/logstash-filter-translate maybe they know more.

11

No, the value of dictionary_path is passed straight to IO.read, it is not sprintf'd first.

12

i see..

is it possible to use nested dictionaries with translate to accomplish what i want to do? for example:

apple:
12345: macbook

13

No not that I know.

the only thing you could do is something like this (pseudocode)

if(name == "Apple"){
    translate{..... dictionary_path: "apple.yml"}
}else if (name == "Samsung"){
    translate{.... dictionary_path: "samsung.yml"}
)

you would need to create a lot of translates and if queries.

14

yeah unfortunately I would potentially need to create 1000s

15

Then I would suggest opening an issue at the github repo I linked above.

16

If you're willing to transform your translate data a bit you could structure the file like:

Apple_12345: macbook
Apple_12346: iphone
Dell_12345: laptop

Then construct a new field to use for your translate:

translate {
    add_field => { "lookup" => "%{brand}_%{part_number}" }
    field => "lookup"
    destination => "model"
    dictionary_path => "translate_file.yml"
    remove_field =>  [ "lookup" ]
}
17

thank you that is a good work around I will have to try it

18

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.