trying to transfer data in json format which in external service to elasticseach.
data provide page by page. next page define in previous page
how to transfer data from all pages to elasticseach
sample json file
count: 565946
next: "http://test.data.tech/api/static/v1/hotels/?format=json&page=2"
previous: "null"
results
0: {…}
1:
address: "Keizersgracht 164"
availability_score: 0.000837698434291508
book_score: 501
checkin_from: "14:00"
used http_poller input plugin but it transfer only data in first page.
input {
http_poller {
urls => {
test2 => {
method => GET
user => "xxxx"
password => "xxxx"
url => "http://test.data.tech/api/static/v1/hotels"
}
}
request_timeout => 60
schedule => { cron => "* * * * * UTC"}
codec => "json"
}
}
Is there way to transfer data in all pages to a index