Pagination for REST API Through Logstash

How can I import from a REST API to Elasticsearch via Logstash using Pagination?

I was able to import using http_poller but unfortunately, it doesn't support pagination, so how can I write force a custom script or "for loop" to check for other records to import, as each API call retrieves only 50 records and I have more than 100 K.

Please help how can I apply pagination logic here.


I think there's an issue for support for that in the plugin repo, but it doesn't support it at this stage unfortunately.

Thank you Mark for your response.
Is there any other way, if http_poller doesn't support pagination. Like using exec command or any other plugin?


This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.