Pull logs from filebeat instead of pushing by filebeat

[Sohrab.p72 (Sohrab Panahandeh)]

Hi,
I need elk to pull data from filebeat it means I don't want the data be pushed by Filebeat to any endpoint.
Like Node_exporter for Prometheus which is listening on an endpoint for prometheus, I want filebeat or any other Beats to listen for elasticsearch requests to pull data from.
Can anyone help with the concept in any way?
Thanks in advance.

[stephenb (Stephen Brown)]

Hi @Sohrab.p72 Welcome to the community!

First, Elasticsearch does not natively pull/poll data nor issue request to pull data.

In general beats push data to elasticsearch but there are beats and logstash "inputs" that can pull data from a source and then push to elasticsearch

Examples are filebeat http input

Or logstash http input

Metricbeat Prometheus support.

And of course there are many ETL tools that support elasticsearch as a sink.

Perhaps if you provide a little more about your use cases we might be able to help.

Elasticsearch has a well documented REST endpoint to ingest data

But again Elasticsearch does not natively pull data.

[Sohrab.p72 (Sohrab Panahandeh)]

Thanks for your quick reply.
Actually my elasticsearch nodes are my PC at home and my loptop at work. They are both behind firewall and routers and portforwarding and D-NAT cost me more than finding a pull/poll model😁
Anyway, I'm newbe in ELK and got some servers for my own, not at work, to have Beats on and monitor them.
I'm just looking around, not an enterprise user. Just want to give new things a try.