I have to bulk import data via LogStash as below:
Source: Restful Get APIs
Destination : Elasticsearch
My .conf file looks as below:
input {
http_poller {
urls => {
ticket_status => {
method => get
url => "https://someurl/movies?page=0&size=30"
headers => {
"accept" => "application/json"
}
}
}
This fetches 30 records at a time.
But, i have more than 5000 records and need to apply pagination logic in input of http-poller.
Please help how can i apply pagination logic here.
If not possible then please let me know how to do it custom pulgin.
Thanks.