Hi, I have an implementation of some script that calls an api to start some job, I can follow up the job status another api (jobs-api). I want to use Logstash and Elasticsearch to monitor the status of the jobs I start, in order to do that I am using http_poller input plugin.
input {
http_poller {
urls => {
test1 => "http://somehost/jobs?jobsLabel=nameOfApp_20201228"
}
request_timeout => 60
# Supports "cron", "every", "at" and "in" schedules by rufus scheduler
schedule => { cron => "0 * * * * UTC"}
codec => "json"
# A hash of request metadata info (timing, response headers, etc.) will be sent here
metadata_target => "http_poller_metadata"
}
}
As you can see in the urls part the endpoint follows the pattern:
http://somehost/jobs?jobsLabel=sometext_yyyyMMdd
Where nameOfApp_20201228 is some label my script creates when it starts the job, and it contains the name of the app followed by the date of the day when the job has been created in this example it is 20201228.
Is there a way to write the config file of logstash that dynamically updates the jobsLabel to be today's date? or at least to watch all job-labels starts with nameOfApp_? something like:
Yes, I'm one of them looking for that solid solution
for the moment i generate dynamically the configuration that it get loaded everytime updated.
May be @Badger and @leandrojmp an advice on this topic
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.