Logs Pulling Architecture

Elastic Stack Elasticsearch
,
1

Filebeat uses a "push" architecture, sending logs to logstash or elastic. Is there a feature or product that would enable elastic to receive hosts' logs using a "pull" architecture?

In my setup, I have network connectivity only from my elasticsearch cluster to the hosts I would like to collect the logs from, so I can't use the default log shipping architecture of filebeat or fluentbit.

Thanks!

2

Hi,

Elastic has some options for pulling logs:
We use the logstash jdbc input to pull logs from our databases.
We use the logstash http_poller input to get the current state of some machines.

For logs, your best way - although not strictly a pull architecture - might be to use a message bus:
You decouple the production from the logging system as both systems only talk to the messagebus server (e.g. Kafka). Filebeat already supports Kafka for both input and output, so you could run a FileBeat on the production system sending the logs and a FileBeat instance on the elasticsearch cluster consuming them.

Best regards
Wolfram

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.