How to use a dynamic value (from shell?) in input { } configuration?

Elastic Stack Logstash
1

I'm testing logstash for the first time, and I'm using it to get data from a database.

I've already managed to configure jdbc-input pluging, and I'm using a query that uses a date as a parameter.

...
parameters => { "date" => "20170417" }
statement => "SELECT id, date, doc, value FROM table WHERE date = :data"
...

I'd need to get this date parameter from the current date, and I thought I could use a bash script like date +%Y%m%d to get this date.

Is that possible, or is there any other solution I can use ?

It would be something like this

...
parameters => { "date" => $(date +%Y%m%d) }
statement => "SELECT id, date, doc, value FROM table WHERE date = :data"
...
2

In many cases you can use ${varname} to reference environment variables but you can't get Logstash to run a command like in your example. Why do you need the current date? Maybe there's another way of accomplishing your end goal.

3

Maybe %{varname} can help in most cases.

For the current date, in this case that I'm using jdbc, I can get it via T-SQL.

4

@magnusbaeck I thought of a case where I could need to get the output of a shell command.

I've been using logstash's docker image, and recente versions of Docker has a docker secrets option, so I can save passwords and other sensitive information.

Tha way docker makes it available to containers is via a tmpfs filesystem at /run/secrets directory, and won't use environment variables anymore.

So if I need a password created via docker secret create mypassword I need to get its content via cat /run/secrets/mypassword

This should be a nice example of the need to get the output of a shell command.

See : https://www.diogomonica.com/2017/03/27/why-you-shouldnt-use-env-variables-for-secret-data/

Ok, in my input-jdbc case I know I have the jdbc_password_filepath, but think other scenarios. :wink: