Pagination for REST API Through Logstash

Elastic Stack Logstash
1

How can I import from a REST API to Elasticsearch via Logstash using Pagination?

I was able to import using http_poller but unfortunately, it doesn't support pagination, so how can I write force a custom script or "for loop" to check for other records to import, as each API call retrieves only 50 records and I have more than 100 K.

Please help how can I apply pagination logic here.

Thanks.

2

I think there's an issue for support for that in the plugin repo, but it doesn't support it at this stage unfortunately.

3

Thank you Mark for your response.
Is there any other way, if http_poller doesn't support pagination. Like using exec command or any other plugin?

Regards

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.