Runing logstash service with number of config files

Elastic Stack Logstash
1

Hi All,
I'm running logstash as a service on ubuntu machine with 10 different config files.each config get different doc type from different s3 bucket and insert it or delete it from elasticsearch.
Yesterday i saw that logstash is stuck on specific config that took him 1 day to complete and because of that non of the other configs was started.
what should i do? run 10 different Crontab jobs?

10x.

2

Try using the -w flag to give Logstash more workers (ie threads).

3

The problem is that my elasticsearch nodes are at 100% cpu.
Can't i tell logstash to switch configuration every file input in s3?

4

Sounds like you need more resources then, there's not much I can think of at the moment to help here sorry.

5

@karasmeitar

Hi , just wanted to know if you have solved the running of many config files ? I have a similar situation where i have to run many config files with some scheduling involved. Is it possible ?

Thanks in advance :slight_smile: