200 pipelines in logstash causing scalability issues


I need a design suggestion.

We have 200 microservices and a 4 node logstash HA setup.
Now each microservice has to send logs to a different index. Current setup I have is, for each microservice we have a different pipeline in logstash on a particular port, and that port is mentioned in filebeat for that service.

This has caused 200 pipelines to come up in logstash which is causing issues and highly unscalable.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.