# Multiple Logstash Instance in one machine

**URL:** <https://discuss.elastic.co/t/multiple-logstash-instance-in-one-machine/96717>\
**Category:** Logstash\
**Created:** [August 11, 2017, 7:34am UTC](https://discuss.elastic.co/t/multiple-logstash-instance-in-one-machine/96717 "2017-08-11T07:34:49Z")\
**Posts on this page:** 19\
**Page:** 1

<div class="post-metadata">

**Author:** ![jogoinar10](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jogoinar10/32/20068_2.png) [@jogoinar10](https://discuss.elastic.co/u/jogoinar10)\
**Post date:** [August 11, 2017, 7:34am UTC](https://discuss.elastic.co/t/multiple-logstash-instance-in-one-machine/96717/1 "2017-08-11T07:34:49Z")

</div>

How can I setup multiple logstash instance in one machine?  
My idea is to gather logs and then forward the data to two elasticsearch host machine as well.  
If you'll ask why not to config in just single logstash with 2 elasticsearch output it that I don't want to stop the sending of data if one of the elasticsearch fail.

Or is there a way to just forward the data from 1 elasticsearch host to another elasticsearch host?

TIA

---

<div class="post-metadata">

**Author:** ![jogoinar10](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jogoinar10/32/20068_2.png) [@jogoinar10](https://discuss.elastic.co/u/jogoinar10)\
**Post date:** [August 18, 2017, 12:52am UTC](https://discuss.elastic.co/t/multiple-logstash-instance-in-one-machine/96717/2 "2017-08-18T00:52:11Z")

</div>

up for this. i still need help

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [August 18, 2017, 3:02am UTC](https://discuss.elastic.co/t/multiple-logstash-instance-in-one-machine/96717/3 "2017-08-18T03:02:59Z")

</div>

There's an answer [here](https://discuss.elastic.co/t/solved-multiple-instances-for-logstash/65539/2?u=theuntergeek) and [here](https://discuss.elastic.co/t/setting-jvm-options-with-multiple-logstash-5-x-instances-on-the-same-box/67765).

---

<div class="post-metadata">

**Author:** ![jogoinar10](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jogoinar10/32/20068_2.png) [@jogoinar10](https://discuss.elastic.co/u/jogoinar10)\
**Post date:** [August 18, 2017, 7:44am UTC](https://discuss.elastic.co/t/multiple-logstash-instance-in-one-machine/96717/4 "2017-08-18T07:44:57Z")

</div>

> If you plan on monitoring Logstash, you may want to manually set the http.port and/or http.host so you know which instance is which.

is it important to set the http.port? or http.post?

---

<div class="post-metadata">

**Author:** ![jogoinar10](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jogoinar10/32/20068_2.png) [@jogoinar10](https://discuss.elastic.co/u/jogoinar10)\
**Post date:** [August 18, 2017, 8:19am UTC](https://discuss.elastic.co/t/multiple-logstash-instance-in-one-machine/96717/5 "2017-08-18T08:19:25Z")

</div>

i already tried this guide of yours [[SOLVED] Multiple Instances for Logstash](https://discuss.elastic.co/t/solved-multiple-instances-for-logstash/65539/2?u=theuntergeek) but it doesn't work for me. I tried to check the logstash log but there's no logfiles

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [August 18, 2017, 4:07pm UTC](https://discuss.elastic.co/t/multiple-logstash-instance-in-one-machine/96717/6 "2017-08-18T16:07:05Z")

</div>

What didn't work for you? I follow this exact recipe at home, and it works flawlessly.

Please list your exact steps so I can follow your workflow.

---

<div class="post-metadata">

**Author:** ![jogoinar10](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jogoinar10/32/20068_2.png) [@jogoinar10](https://discuss.elastic.co/u/jogoinar10)\
**Post date:** [August 18, 2017, 6:00pm UTC](https://discuss.elastic.co/t/multiple-logstash-instance-in-one-machine/96717/7 "2017-08-18T18:00:31Z")

</div>

I'm currently using logstash 5.5.  
First i just make a copy of my logstash. So it will reside at `/etc/logstash2`.  
Make a data path `/var/lib/logstash2` ang log path `/var/log/logstash2`.  
Then edit the yml config.  
On the startup options I change the `LS_SETTINGS_DIR` to `/etc/logstash2`.  
Change the `SERVICE_NAME and SERVICE_DESCRIPTION to logstash2`.  
I run the`/bin/system-install /etc/logstash2` \<--i don't know if this is right.  
Then run the `service logstash2 start` command.

I run successfully based on the `service logstash2 status`.  
BUT, there's no logs produced in `/var/log/logstash2` and there's no data was sent to my output which is elastissearch.

After which i tried to run my main logstash but it ended up broke as well. On the log of my main logstash, it says that it cannot find the config file or config file doesn't exist.

So, i uninstall my main logstash thru `yum remove logstash` and install it again. Then my main logstash came back to work again.

Sadly, I just cant run multiple logstash as of this moment. 😥

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [August 18, 2017, 7:20pm UTC](https://discuss.elastic.co/t/multiple-logstash-instance-in-one-machine/96717/8 "2017-08-18T19:20:41Z")

</div>

Did you make any other changes to the `logstash.yml` file in `/etc/logstash2`?

Particularly, did you give your second logstash a different `node.name`? And `path.config`?

```auto
$ grep -v ^# logstash.yml | grep [a-z]
node.name: logstash_output
path.data: /var/lib/logstash/output
path.config: /etc/logstash/output/conf.d
config.reload.automatic: true
path.logs: /var/log/logstash/output

```

---

<div class="post-metadata">

**Author:** ![jogoinar10](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jogoinar10/32/20068_2.png) [@jogoinar10](https://discuss.elastic.co/u/jogoinar10)\
**Post date:** [August 19, 2017, 12:36am UTC](https://discuss.elastic.co/t/multiple-logstash-instance-in-one-machine/96717/9 "2017-08-19T00:36:47Z")

</div>

i just change the `path.data`, `path.logs`, and `path.config`  
i didn't change the [node.name](http://node.name)

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [August 19, 2017, 7:44pm UTC](https://discuss.elastic.co/t/multiple-logstash-instance-in-one-machine/96717/10 "2017-08-19T19:44:13Z")

</div>

Not changing the `node.name` will result in a conflict. That node wouldn't be able to start.

---

<div class="post-metadata">

**Author:** ![jogoinar10](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jogoinar10/32/20068_2.png) [@jogoinar10](https://discuss.elastic.co/u/jogoinar10)\
**Post date:** [August 20, 2017, 4:36am UTC](https://discuss.elastic.co/t/multiple-logstash-instance-in-one-machine/96717/11 "2017-08-20T04:36:11Z")

</div>

alright. i'll try to change the `node.name`. Thank you very much!

---

<div class="post-metadata">

**Author:** ![jogoinar10](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jogoinar10/32/20068_2.png) [@jogoinar10](https://discuss.elastic.co/u/jogoinar10)\
**Post date:** [August 22, 2017, 3:48am UTC](https://discuss.elastic.co/t/multiple-logstash-instance-in-one-machine/96717/12 "2017-08-22T03:48:42Z")

</div>

> [@theuntergeek](#):
>
> config.reload.automatic: true

What is the use of this config reload automatic?

Btw, I was able to start logstash by changing the node.name.  
Thanks!

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [August 22, 2017, 3:25pm UTC](https://discuss.elastic.co/t/multiple-logstash-instance-in-one-machine/96717/13 "2017-08-22T15:25:24Z")

</div>

`config.reload.automatic: true` means that if I edit the logstash configuration, the updated pipeline will automatically reload with the new configuration, without having to stop/restart logstash.

---

<div class="post-metadata">

**Author:** ![jogoinar10](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jogoinar10/32/20068_2.png) [@jogoinar10](https://discuss.elastic.co/u/jogoinar10)\
**Post date:** [August 23, 2017, 4:52am UTC](https://discuss.elastic.co/t/multiple-logstash-instance-in-one-machine/96717/14 "2017-08-23T04:52:51Z")

</div>

Thanks for explaining @theuntergeek! 💯

---

<div class="post-metadata">

**Author:** ![jogoinar10](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jogoinar10/32/20068_2.png) [@jogoinar10](https://discuss.elastic.co/u/jogoinar10)\
**Post date:** [August 23, 2017, 6:00am UTC](https://discuss.elastic.co/t/multiple-logstash-instance-in-one-machine/96717/15 "2017-08-23T06:00:33Z")

</div>

Upon making another setup to another server. i got this error.

`[WARN][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because 'path.config' (-f) is being used.`

It's weird since I didn't got this error during my setup on the first one.

---

<div class="post-metadata">

**Author:** ![Mojster](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mojster/32/21209_2.png) [@Mojster](https://discuss.elastic.co/u/Mojster)\
**Post date:** [August 23, 2017, 6:56am UTC](https://discuss.elastic.co/t/multiple-logstash-instance-in-one-machine/96717/16 "2017-08-23T06:56:03Z")

</div>

> [@jogoinar10](#):
>
> How can I setup multiple logstash instance in one machine?
> 
> My idea is to gather logs and then forward the data to two elasticsearch host machine as well.
> 
> If you’ll ask why not to config in just single logstash with 2 elasticsearch output it that I don’t want to stop the sending of data if one of the elasticsearch fail.
> 
> Or is there a way to just forward the data from 1 elasticsearch host to another elasticsearch host?
> 
> TIA

On Windows server it was pretty simple for me. 🙂

I've just extracted LS to a new location and in the pipeline I've set to listen to a different port.  
I'm currently running 3 instances on the same server.

---

<div class="post-metadata">

**Author:** ![jogoinar10](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jogoinar10/32/20068_2.png) [@jogoinar10](https://discuss.elastic.co/u/jogoinar10)\
**Post date:** [August 23, 2017, 6:57am UTC](https://discuss.elastic.co/t/multiple-logstash-instance-in-one-machine/96717/17 "2017-08-23T06:57:32Z")

</div>

Im running on centos. 🙂 It was just simple setup in some other server. but i don't know why there's some error on others.

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [August 23, 2017, 11:52am UTC](https://discuss.elastic.co/t/multiple-logstash-instance-in-one-machine/96717/18 "2017-08-23T11:52:48Z")

</div>

That's a warning, not an error. There's nothing wrong with that message.

However, it suggests you're using different release versions. pipelines.yml is not in 5.5.2, but only in newer beta releases of 6.0

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 20, 2017, 11:53am UTC](https://discuss.elastic.co/t/multiple-logstash-instance-in-one-machine/96717/19 "2017-09-20T11:53:03Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.