Pagination Logic in input of HTTP Poller plugin, if not how to do it in custom plugin

I have to bulk import data via LogStash as below:
Source: Restful Get APIs
Destination : Elasticsearch

My .conf file looks as below:
input {
http_poller {
urls => {
ticket_status => {
method => get
url => "https://someurl/movies?page=0&size=30"
headers => {
"accept" => "application/json"
}
}
}

This fetches 30 records at a time.
But, i have more than 5000 records and need to apply pagination logic in input of http-poller.

Please help how can i apply pagination logic here.

If not possible then please let me know how to do it custom pulgin.

Thanks.

You have set size 30 in url. Check documentation which params do you need. Most likely need to remove size.

ticket_status => {
method => get
url => "https://someurl/movies?page=0&size=30"

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.