# RESTful APIs and http\_poller input plugin

**URL:** https://discuss.elastic.co/t/restful-apis-and-http-poller-input-plugin/169450
**Category:** Logstash
**Created:** [February 21, 2019, 3:35pm UTC](https://discuss.elastic.co/t/restful-apis-and-http-poller-input-plugin/169450 "2019-02-21T15:35:55Z")
**Posts on this page:** 2
**Page:** 1

<div class="post-metadata">

### Author: ![nandrik](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nandrik/32/39951_2.png) [@nandrik](https://discuss.elastic.co/u/nandrik)
#### Post date: [February 21, 2019, 3:35pm UTC](https://discuss.elastic.co/t/restful-apis-and-http-poller-input-plugin/169450/1 "2019-02-21T15:35:55Z")

</div>

I've found myself in a few situations where there is a RESTful API that I want to consume and which updates over time with newer entries.

I wonder what's the best way to be periodically probing the API and consistently getting all the records without missing any of them?

I realize http\_poller is stateless. Would the safest way be to always be querying a larger block of entries and deduplicating them through unique identifiers?

Alternatively, what I've been doing so far is write API apps that consume the API and output a log file that I can then use Filebeat to send to Logstash.

Wonder what are the best practices when it comes to RESTful APIs.

**Update** : Since I first wrote this post a couple of months ago, I've resorted into client-side scripting and have written an updated post with my more detailed thoughts on the **pros** and **cons** between **client- and Logstash- side coding** , see: [REST API, http\_poller and ruby code](https://discuss.elastic.co/t/rest-api-http-poller-and-ruby-code/176447)

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 21, 2019, 3:35pm UTC](https://discuss.elastic.co/t/restful-apis-and-http-poller-input-plugin/169450/2 "2019-03-21T15:35:55Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
