I have to bulk import data via LogStash as below:
Source: Restful Get APIs
Destination : ElasticSearch
My .conf file looks as below:
input {
http_poller {
urls => {
test1 => {
method => get
url => "https://someurl/movies?limit=100&offset=0"
headers => {
Accept => "application/json"
}
}
}
This fetches 100 records at a time.
But, i have more than 5000 records and need to apply pagination logic in input of http-poller.
Please help how can i apply pagination logic here.
Thanks.