How can I setup multiple logstash instance in one machine?
My idea is to gather logs and then forward the data to two elasticsearch host machine as well.
If you'll ask why not to config in just single logstash with 2 elasticsearch output it that I don't want to stop the sending of data if one of the elasticsearch fail.
Or is there a way to just forward the data from 1 elasticsearch host to another elasticsearch host?
TIA
up for this. i still need help
If you plan on monitoring Logstash, you may want to manually set the http.port and/or http.host so you know which instance is which.
is it important to set the http.port? or http.post?
i already tried this guide of yours [SOLVED] Multiple Instances for Logstash but it doesn't work for me. I tried to check the logstash log but there's no logfiles
What didn't work for you? I follow this exact recipe at home, and it works flawlessly.
Please list your exact steps so I can follow your workflow.
I'm currently using logstash 5.5.
First i just make a copy of my logstash. So it will reside at /etc/logstash2.
Make a data path /var/lib/logstash2 ang log path /var/log/logstash2.
Then edit the yml config.
On the startup options I change the LS_SETTINGS_DIR to /etc/logstash2.
Change the SERVICE_NAME and SERVICE_DESCRIPTION to logstash2.
I run the/bin/system-install /etc/logstash2 <--i don't know if this is right.
Then run the service logstash2 start command.
I run successfully based on the service logstash2 status.
BUT, there's no logs produced in /var/log/logstash2 and there's no data was sent to my output which is elastissearch.
After which i tried to run my main logstash but it ended up broke as well. On the log of my main logstash, it says that it cannot find the config file or config file doesn't exist.
So, i uninstall my main logstash thru yum remove logstash and install it again. Then my main logstash came back to work again.
Sadly, I just cant run multiple logstash as of this moment. 
Did you make any other changes to the logstash.yml file in /etc/logstash2?
Particularly, did you give your second logstash a different node.name? And path.config?
$ grep -v ^# logstash.yml | grep [a-z]
node.name: logstash_output
path.data: /var/lib/logstash/output
path.config: /etc/logstash/output/conf.d
config.reload.automatic: true
path.logs: /var/log/logstash/output
i just change the path.data, path.logs, and path.config
i didn't change the node.name
Not changing the node.name will result in a conflict. That node wouldn't be able to start.
alright. i'll try to change the node.name. Thank you very much!
What is the use of this config reload automatic?
Btw, I was able to start logstash by changing the node.name.
Thanks!
config.reload.automatic: true means that if I edit the logstash configuration, the updated pipeline will automatically reload with the new configuration, without having to stop/restart logstash.
Thanks for explaining @theuntergeek! 
Upon making another setup to another server. i got this error.
[WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because 'path.config' (-f) is being used.
It's weird since I didn't got this error during my setup on the first one.
On Windows server it was pretty simple for me. 
I've just extracted LS to a new location and in the pipeline I've set to listen to a different port.
I'm currently running 3 instances on the same server.
Im running on centos. 
It was just simple setup in some other server. but i don't know why there's some error on others.
That's a warning, not an error. There's nothing wrong with that message.
However, it suggests you're using different release versions. pipelines.yml is not in 5.5.2, but only in newer beta releases of 6.0
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
© 2020. All Rights Reserved - Elasticsearch
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.