Need to frequently change url-timestamp value within logstash input

Elastic Stack Logstash
1

Hi,
I'm parsing some json data from a website using rest api with start_timestamp of 4 hours before current time period. Is there a way within logstash to write such code to get data from last 4 hours on every runtime or is python script the way to go?

Logstash config will run every 5 minutes or so and the input url needs to have its timestamp value changes accordingly.
urlname => "https://targetIP/api/v1/alerts?start_timestamp=2022-09-09T00:00:00.000Z+01:00"

logstash config:

input{
	http_poller{
		urls =>{
		    urlname => "https://targetIP/api/v1/alerts?start_timestamp=2022-09-09T00:00:00.000Z+01:00"
        }
		user => "user1"
        password => "pwd123"
		request_timeout => 60
		schedule => {every => "30s"}
		codec => "json"
		ssl_verification_mode => "none"
	}
	
}
# Working version: pls do not edit without approval
output{
	elasticsearch{
		hosts => ["https://IP:9200"]
		index => "forescout-alerts"
		ssl => true
		ssl_certificate_verification => false
		user => "uname"
    	password => pwdpwd"
	}

	stdout {codec => rubydebug}
}
2

You could use a heartbeat input to control the schedule, then use an http filter to make the request.

3

I'm using python script to calculate the latest timestamp and script is changing the url value. https://targetIP/api/v1/alerts?start_timestamp=2022-09-09T00:00:00.000Z+01:00". Is there a way http_poller can use the py output as url name?

4

No, there is not.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.