Logstash translate plugin dynamic dictionary path

Elastic Stack Logstash
1

Hi,

I am creating a csv file which I want to use as a dictionary inside translate plugin.
Since the mapping might change the next day, I named my csv file this way:

my-mapping-YYYY-MM-dd.csv

where YYYY, MM, dd are year, month, date respectively.

I would like to read the latest csv file through the translate plugin. So, the logstash configuration used:

translate {
    field => "id"
    destination => "name"
    dictionary_path => "/home/my-mapping-%{+YYYY-MM-dd}.csv"
}

But, looks like dictionary_path parameter is unable to resolve %{+YYYY-MM-dd}
This is the error I got with it:

Invalid setting for translate filter plugin:
filter {
    translate {
      # This setting must be a path
      # File does not exist or cannot be opened /home/my-mapping-%{+YYYY-MM-dd}.csv
      dictionary_path => "/home/my-mapping-%{+YYYY-MM-dd}.csv"

Any workaround for this problem?

Thanks

2

As a workaround, I am able to achieve this using Environment variables.

export DATE=$(date +%F)

Then inside config:

dictionary_path => "/home/my-mapping-${DATE}.csv"

Any better solutions are welcome.

3

Your solution obviously requires you to restart Logstash every day but apart from that it's probably the best available solution at this point.

4

Thanks for suggesting about restarting logstash. Didn't occur to mind.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.