# How should i setup logstash to collect from multiple machines?

**URL:** <https://discuss.elastic.co/t/how-should-i-setup-logstash-to-collect-from-multiple-machines/52750>\
**Category:** Logstash\
**Created:** [June 14, 2016, 2:55pm UTC](https://discuss.elastic.co/t/how-should-i-setup-logstash-to-collect-from-multiple-machines/52750 "2016-06-14T14:55:08Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![dal](https://avatars.discourse-cdn.com/v4/letter/d/ebca7d/32.png) [@dal](https://discuss.elastic.co/u/dal)\
**Post date:** [June 14, 2016, 2:55pm UTC](https://discuss.elastic.co/t/how-should-i-setup-logstash-to-collect-from-multiple-machines/52750/1 "2016-06-14T14:55:08Z")

</div>

I am new to ES and Logstash and I was trying to setup logstash to collect logs from specific directories and forward them to my Elasticsearch instance. My ES node runs on my local machine and the logs are located in different directories on different VMs.

My question is, what is the ideal setup configuration in this case? I was looking at this page:

[https://www.elastic.co/guide/en/logstash/current/advanced-pipeline.html#configuring-geoip-plugin](https://www.elastic.co/guide/en/logstash/current/advanced-pipeline.html#configuring-geoip-plugin)

which discusses how to set up a config file and I am not sure whether I should run a single logstash instance on my local machine and setup filebeats on the VMs or should I have multiple logstash instances (one for each VM) and forward that to my ES node. Is there even a need for filebeats?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 14, 2016, 8:05pm UTC](https://discuss.elastic.co/t/how-should-i-setup-logstash-to-collect-from-multiple-machines/52750/2 "2016-06-14T20:05:37Z")

</div>

Either method works. I prefer doing as little as possible on the leaf nodes and having them send all events to a central Logstash server (or set of servers) that does all the filtering.

If the VMs are small the much bigger overhead of Logstash makes Filebeat a better option.

---

<div class="post-metadata">

**Author:** ![dal](https://avatars.discourse-cdn.com/v4/letter/d/ebca7d/32.png) [@dal](https://discuss.elastic.co/u/dal)\
**Post date:** [June 14, 2016, 8:09pm UTC](https://discuss.elastic.co/t/how-should-i-setup-logstash-to-collect-from-multiple-machines/52750/3 "2016-06-14T20:09:38Z")

</div>

that makes sense, i'll run a local instance of logstash and use filebeats on the vms. thanks for the advice.

---

<div class="post-metadata">

**Author:** ![vinuja](https://avatars.discourse-cdn.com/v4/letter/v/a4c791/32.png) [@vinuja](https://discuss.elastic.co/u/vinuja)\
**Post date:** [August 17, 2016, 8:53am UTC](https://discuss.elastic.co/t/how-should-i-setup-logstash-to-collect-from-multiple-machines/52750/4 "2016-08-17T08:53:08Z")

</div>

Hey dal, I am trying to do the same configuration.

So I each filebeat on my vms as specified:

> filebeat:  
> prospectors:  
> -  
> paths:  
> - \mypath\*.log  
> document\_type: mytype  
> input\_type: log  
> ignore\_older: 0  
> close\_older: 0  
> fields:  
> product: order\_online  
> fields\_under\_root: true  
> tail\_files: false
> 
> output:  
> logstash:  
> hosts: ["My.Local.Machine.ip.adress:5044"]  
> shipper:  
> logging:  
> files:  
> rotateeverybytes: 10485760

And then in my local machine I have the following configuration:

> input {  
> beats {  
> port =\> 5044  
> ssl =\> false

> }  
> }

> filter {...}  
> output {

```
if "_grokparsefailure" in [tags] {
    file {
        path => "mypath2/error.log"
    } 
}else{
	elasticsearch{
		hosts => "My.Local.Machine.ip.adress:9200"
		index => "myindexname"
		manage_template => false
		document_type => "mydocname"
	}
	
	output {
		stdout { codec => json }
	}
	
}

```

}

With this configuration it's collecting from my local machine but not from the VM machines, I think i am missing something, can you please help me ?

Thanks

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 17, 2016, 9:21am UTC](https://discuss.elastic.co/t/how-should-i-setup-logstash-to-collect-from-multiple-machines/52750/5 "2016-08-17T09:21:22Z")

</div>

> ```
> - \mypath*.log
> 
> ```

Are you missing a drive letter here? I also believe you need to quote the whole value, use two backslashes, or use forward slashes.

---

<div class="post-metadata">

**Author:** ![vinuja](https://avatars.discourse-cdn.com/v4/letter/v/a4c791/32.png) [@vinuja](https://discuss.elastic.co/u/vinuja)\
**Post date:** [August 17, 2016, 9:37am UTC](https://discuss.elastic.co/t/how-should-i-setup-logstash-to-collect-from-multiple-machines/52750/6 "2016-08-17T09:37:07Z")

</div>

The path is well specified as I works fine if I just run the logstash and ES in the VM.  
I am sure it's due to the fact that I am trying to connect with an Logstash/ES which are not in the same machine.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 17, 2016, 10:39am UTC](https://discuss.elastic.co/t/how-should-i-setup-logstash-to-collect-from-multiple-machines/52750/7 "2016-08-17T10:39:25Z")

</div>

Well, if Filebeat has problems connecting to Logstash there should be an error message in the Filebeat log.

---

<div class="post-metadata">

**Author:** ![krddy1026](https://avatars.discourse-cdn.com/v4/letter/k/b5a626/32.png) [@krddy1026](https://discuss.elastic.co/u/krddy1026)\
**Post date:** [October 1, 2016, 7:20am UTC](https://discuss.elastic.co/t/how-should-i-setup-logstash-to-collect-from-multiple-machines/52750/8 "2016-10-01T07:20:34Z")

</div>

HI can someone help how to install logstash

I tried

echo "deb [http://packages.elastic.co/elasticsearch/2.x/debian](http://packages.elastic.co/elasticsearch/2.x/debian) stable main" | sudo tee -a /etc/apt/sources.list.d/elasticsearch-2.x.list

echo "deb [http://packages.elastic.co/kibana/4.4/debian](http://packages.elastic.co/kibana/4.4/debian) stable main" | sudo tee -a /etc/apt/sources.list.d/kibana-4.4.x.list

and executed apt-get install logstash

But it was saying unable to locate packages.

Can someone please help me how to fix this issue

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [October 1, 2016, 3:38pm UTC](https://discuss.elastic.co/t/how-should-i-setup-logstash-to-collect-from-multiple-machines/52750/9 "2016-10-01T15:38:05Z")

</div>

@krddy1026, please start a new thread for your completely unrelated question.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:36am UTC](https://discuss.elastic.co/t/how-should-i-setup-logstash-to-collect-from-multiple-machines/52750/10 "2017-07-06T04:36:13Z")

</div>


