Dynamic dictionary?

Elastic Stack Logstash
1

For some reason I can't reference a field in my dictionary_path, is this even possible? I want to use a different dictionary depending on a field.

2

Please show your config, and share what version you are running.

3

I am using 6.2.4 and here's the translation portion of my config file:

translate { 
    field => "metricId"
    dictionary_path => "/Users/user/diagviz/dict/%{[bundle]}.json"
    destination => "metricTemp"
    override => true 

}

Where bundle I know is a field that is defined, but the output doesn't replace it with the field when I get the error message.

This is the error message:
[2018-07-05T14:31:57,713][ERROR][logstash.filters.translate] Invalid setting for translate filter plugin:

filter {
translate {
# This setting must be a path
# File does not exist or cannot be opened /Users/user/diagviz/dict/%{[bundle]}.json
dictionary_path => "/Users/user/diagviz/dict/%{[bundle]}.json"
...
}
}

4

Environment variables do work though.

5

With a quick view in the source code it seems that is not possible

1 Like
6

I suspect you may need to use one translate plugin per dictionary and control which one you use through conditionals. You might also be able to create a single dictionary where the key contains the bundle, so you can have all data in one file.

7

The thing is my dictionaries will be variable (I won't know how many dictionaries I need so I generate them dynamically)

8

eventualy you could try instead of the translate filter to use the jdbc static or streaming filter?

https://www.elastic.co/guide/en/logstash/current/plugins-filters-jdbc_static.html
https://www.elastic.co/guide/en/logstash/current/plugins-filters-jdbc_streaming.html

using these with a sqlite db as source would probaly fill your needs

1 Like
9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.