Running multiple pipelines and a single pipeline triggered with shell script simultaneously

Elastic Stack Logstash
1

Hi.

I have 5 pipelines already running on linux server.
The conf files are listed in a pipeline.yml and it has already been started with sudo systemctl start logstash, up and running.

I created another single pipeline (single.conf) and a shell script which triggers execution of single.conf. I want to set a cron job and execute shell script once in a day.

Now the problem is, if i start shell script, it triggers single.conf but logstash is already running and it doesn't work.

Do you have any suggestions?
Thanks.

2

I solved this problem and it works fine.

Cron job triggers shellscript.sh

** * * * export LOGSTASH_KEYSTORE_PASS=your_pass_here && cd /etc/logstash/config/ && /bin/bash .sh > /etc/logstash/config/output.log 2>&1

And I defined my single.conf pipeline in shellscript:

"${LOGSTASH_BIN}" -f "${LOGSTASH_CONFIG}" --path.data="${DATA_PATH}"

It is important to create a folder as a new path.data for this purpose and give the rights with chmod +x.

Logstash uses /logstash/data as default even if path.data is not set in logstash.yml.

In single.conf, you can set your normal input, filter and output.

1 Like
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.