# How to transfer logs from different VM's (Digitalocen) to a single Logstash using Filebeat

**URL:** <https://discuss.elastic.co/t/how-to-transfer-logs-from-different-vms-digitalocen-to-a-single-logstash-using-filebeat/222847>\
**Category:** Logstash\
**Created:** [March 10, 2020, 4:04am UTC](https://discuss.elastic.co/t/how-to-transfer-logs-from-different-vms-digitalocen-to-a-single-logstash-using-filebeat/222847 "2020-03-10T04:04:00Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Pathshala\_Gopi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pathshala_gopi/32/45595_2.png) [@Pathshala\_Gopi](https://discuss.elastic.co/u/Pathshala_Gopi)\
**Post date:** [March 10, 2020, 4:04am UTC](https://discuss.elastic.co/t/how-to-transfer-logs-from-different-vms-digitalocen-to-a-single-logstash-using-filebeat/222847/1 "2020-03-10T04:04:00Z")

</div>

Hi all,

From Last two days i got stuck in this task.Task is there are multiple Ubuntu machines are running on cloud (DigitalOcen), I have to take the logs of those machines and ship those logs to Logstash where complete ELK is configured.

I have configured filebeat in one system and my filebeat.yml is like below:

filebeat.prospectors:

- type: log  
paths:
  - /var/log/nginx/_.log  
filebeat.config.modules:  
path: ${path.config}/modules.d/_.yml  
reload.enabled: true  
output.logstash:  
hosts: ["206.189.129.234:5044"]

Logstash:-

And my simple logstash.conf file is like Below

input {  
beats {  
port =\> 5044  
}

}

output {

```
  elasticsearch {
                    hosts => "206.189.129.234:9200"
                    manage_template => false
                    index => "nginx-%{+YYYY.MM.dd}"

```

}

}

when i start the logstash it is running successfully but I am not able to see any index in elasticsearch. I had tried multiple ways but no results can anyone help me Out of this.

And Is there any particular process is there for above scenario..

Thanks in advance....

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [March 10, 2020, 4:36am UTC](https://discuss.elastic.co/t/how-to-transfer-logs-from-different-vms-digitalocen-to-a-single-logstash-using-filebeat/222847/2 "2020-03-10T04:36:59Z")

</div>

Please format your code/logs/config using the `</>` button, or markdown style back ticks. It helps to make things easy to read which helps us help you 🙂 You may also want to redact your public IPs.

Do the filebeat logs show it connecting to Logstash?

---

<div class="post-metadata">

**Author:** ![Pathshala\_Gopi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pathshala_gopi/32/45595_2.png) [@Pathshala\_Gopi](https://discuss.elastic.co/u/Pathshala_Gopi)\
**Post date:** [March 10, 2020, 4:52am UTC](https://discuss.elastic.co/t/how-to-transfer-logs-from-different-vms-digitalocen-to-a-single-logstash-using-filebeat/222847/3 "2020-03-10T04:52:14Z")

</div>

Hi Thanks for reply and suggestion,

I check my file beat logs /var/log/filebeat folder and the logs are

2020-03-10T03:28:07.717Z INFO instance/beat.go:468 Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]

2020-03-10T03:28:07.717Z INFO instance/beat.go:475 Beat UUID: 8a1fc2dc-b9a7-4d1b-ba72-9de650887499

This is the only stuff i have there

---

<div class="post-metadata">

**Author:** ![Pathshala\_Gopi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pathshala_gopi/32/45595_2.png) [@Pathshala\_Gopi](https://discuss.elastic.co/u/Pathshala_Gopi)\
**Post date:** [March 10, 2020, 4:53am UTC](https://discuss.elastic.co/t/how-to-transfer-logs-from-different-vms-digitalocen-to-a-single-logstash-using-filebeat/222847/4 "2020-03-10T04:53:24Z")

</div>

I think filebeat is not connected and shipping the logs to logstash

---

<div class="post-metadata">

**Author:** ![Pathshala\_Gopi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pathshala_gopi/32/45595_2.png) [@Pathshala\_Gopi](https://discuss.elastic.co/u/Pathshala_Gopi)\
**Post date:** [March 10, 2020, 5:40am UTC](https://discuss.elastic.co/t/how-to-transfer-logs-from-different-vms-digitalocen-to-a-single-logstash-using-filebeat/222847/5 "2020-03-10T05:40:28Z")

</div>

Hi when i again start the filebeat i getting the error like below

\<2020-03-10T05:33:22.567Z INFO pipeline/output.go:95 Connecting to backoff(async(tcp://206.189.129.234:5044))/\>  
\<2020-03-10T05:33:31.545Z ERROR fileset/factory.go:105 Error creating input: Can only start an input when all related states are finished: {Id:262590-64513 Finished:false Fileinfo:0xc420518dd0 Source:/var/log/nginx/access.log Offset:0 Timestamp:2020-03-10 05:33:21.544219089 +0000 UTC m=+0.092800296 TTL:-1ns Type:log Meta:map FileStateOS:262590-64513} /\>  
\< 2020-03-10T05:33:31.548Z ERROR [reload] cfgfile/list.go:96 Error creating runner from config: Can only start an input when all related states are finished: {Id:262590-64513 Finished:false Fileinfo:0xc420518dd0 Source:/var/log/nginx/access.log Offset:0 Timestamp:2020-03-10 05:33:21.544219089 +0000 UTC m=+0.092800296 TTL:-1ns Type:log Meta:map FileStateOS:262590-64513} /\>  
\< 2020-03-10T05:33:41.553Z ERROR fileset/factory.go:105 Error creating input: Can only start an input when all related states are finished: {Id:262590-64513 Finished:false Fileinfo:0xc420518dd0 Source:/var/log/nginx/access.log Offset:0 Timestamp:2020-03-10 05:33:21.544219089 +0000 UTC m=+0.092800296 TTL:-1ns Type:log Meta:map FileStateOS:262590-64513} /\>  
\< 2020-03-10T05:33:41.556Z ERROR [reload] cfgfile/list.go:96 Error creating runner from config: Can only start an input when all related states are finished: {Id:262590-64513 Finished:false Fileinfo:0xc420518dd0 Source:/var/log/nginx/access.log Offset:0 Timestamp:2020-03-10 05:33:21.544219089 +0000 UTC m=+0.092800296 TTL:-1ns Type:log Meta:map FileStateOS:262590-64513} /\>

Can you help me out where i am making mistake?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 7, 2020, 5:40am UTC](https://discuss.elastic.co/t/how-to-transfer-logs-from-different-vms-digitalocen-to-a-single-logstash-using-filebeat/222847/6 "2020-04-07T05:40:32Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.