Hi Team,
I working on a setup where i have 5 conf files available on single node cluster(No cluster), the setup was working perfectly fine but off late i could observe that two of my conf files are not getting pickled up regularly for parsing where as the other three parsers are working as expected. I suspect the problem to be with pipeline.workers( where i have allocated 2 workers for each pipeline).
pipelines.yml
- pipeline.id: apache-pipeline
path.config: "/etc/logstash/conf.d/XXXX-apache-pipeline.conf"
pipeline.workers: 2
- pipeline.id: weblog-pipeline
path.config: "/etc/logstash/conf.d/XXXX-websys-pipeline.conf"
pipeline.workers: 2
- pipeline.id: YYYY-apache
path.config: "/etc/logstash/conf.d/YYYY-apache-pipeline.conf"
pipeline.workers: 2
- pipeline.id: YYYY-logs
path.config: "/etc/logstash/conf.d/YYYY-logs-pipeline.conf"
pipeline.workers: 2
- pipeline.id: YYYY-teamlog
path.config: "/etc/logstash/conf.d/YYYY-team-pipeline.conf"
pipeline.workers: 2
Request you to please advice if i am missing anything critical here. Thanks.