Logstash HTTP polling performance


In my use case I need to periodically query huge population of devices(~1M) via HTTP REST API and store status of each device in elasticsearch.

I consider to use http_poller but I worry about perfomance and scalability of this solution. Does anyone have more experience with this input type ?

I suspect Logstash uses one thread per input plugin, so using one http_poller per device clearly won't scale. I don't think a pure Logstash solution is appropriate here. You need something else to do the HTTP polling, but Logstash could still read the results of that polling and process it further.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.