Do we loose backpressure-sensitive protocol by inserting Kafka between Filebeat and Elasticsearch

Elastic Stack Beats
,
1

Hi,
this is a design investigation.
On many docker containers, we will have Filebeat that reads logs (generated by many services).
On one docker container we will have Elasticsearch that receives and indexes the logs.

We will have Kafka in one docker container for other purpose.

  1. Do we have advantages to let Kafka collect logs from the docker containers with Filebeat and forward them to the Elasticsearch container ?
  2. The Filebeat uses a backpressure-sensitive protocol when sending data to Elasticsearch. Won't we loose this advantage by putting Kafka in the middle ?

Thanks for your answers,
Yves

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.