Dynamic dictionary?


(Alex) #1

For some reason I can't reference a field in my dictionary_path, is this even possible? I want to use a different dictionary depending on a field.


(Mark Walkom) #2

Please show your config, and share what version you are running.


(Alex) #3

I am using 6.2.4 and here's the translation portion of my config file:

translate { 
    field => "metricId"
    dictionary_path => "/Users/user/diagviz/dict/%{[bundle]}.json"
    destination => "metricTemp"
    override => true 

}  

Where bundle I know is a field that is defined, but the output doesn't replace it with the field when I get the error message.

This is the error message:
[2018-07-05T14:31:57,713][ERROR][logstash.filters.translate] Invalid setting for translate filter plugin:

filter {
translate {
# This setting must be a path
# File does not exist or cannot be opened /Users/user/diagviz/dict/%{[bundle]}.json
dictionary_path => "/Users/user/diagviz/dict/%{[bundle]}.json"
...
}
}


(Alex) #4

Environment variables do work though.


(Christian Stockhaus) #5

With a quick view in the source code it seems that is not possible


(Christian Dahlqvist) #6

I suspect you may need to use one translate plugin per dictionary and control which one you use through conditionals. You might also be able to create a single dictionary where the key contains the bundle, so you can have all data in one file.


(Alex) #7

The thing is my dictionaries will be variable (I won't know how many dictionaries I need so I generate them dynamically)


(Christian Stockhaus) #8

eventualy you could try instead of the translate filter to use the jdbc static or streaming filter?

https://www.elastic.co/guide/en/logstash/current/plugins-filters-jdbc_static.html
https://www.elastic.co/guide/en/logstash/current/plugins-filters-jdbc_streaming.html

using these with a sqlite db as source would probaly fill your needs


(system) #9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.