Logstash initialize variables before input

[Mario_Morelli]

Hi Guys
I use Logstash for the random projects at work and I just cant seem to figure out the following, seems like it should be easy enough

My input uses the http poller plugin, that posts to a REST API
There are 2 variables that this URL requires

1. Start time
2. End Time

I would just like to understand how I can do the following to input into the http poller
End Time = Need a method to get todays short date 2019-04-12
StartTime = Need a method to get yesterdays short date 2019-04-11

What would be the easiest way to get these variables done, to be used as input variables for the http poller?

Thanks in advance:)

[Badger]

Generate them externally to logstash and use environment variables, or alternatively, see this post.

[alxsss]

How to generate date type variable externally? I see environment variables plugin, but it is a filter plugin not an input plugin.

[Luca_Belluccini]

Hello @Mario_Morelli

Unfortunately http_poller input doesn't support dynamic variables.

I think you can use one of the following approaches.

Use the exec input with an external script

You can use the exec input (which can be scheduled periodically) and use an external script which runs a curl request and prints the output of the call to stdout.

You might need to adjust the codec

input {
  exec {
    # this can be an external script if necessary
    command => 'curl -s -X GET ...'
    schedule => "* * * * *"
  }
}

User the exec input and http filter

You can use the exec input just to trigger the execution periodically.
The command can be a dummy command or a command which returns the date parameters.

Afterwards, use the http filter to perform the call.

input {
  exec {
    command => 'echo `date --date="1 day ago" +"%Y-%m-%d"`,`date +"%Y-%m-%d"`'
    schedule => "* * * * *"
  }
}
filter {
  dissect {
    mapping => {
      "message" => "%{StartTime},%{EndTime}"
    }
    remove_field => "message"
  }
  http {
    ... # use %{StartTime} and %{EndTime}
  }
}

Use environment variables and restart Logstash

If you have to change this daily, you could restart Logstash periodically, changing the value of 2 environment variables prior starting Logstash again.
This is applicable only if you have to change it daily...