From Last two days i got stuck in this task.Task is there are multiple Ubuntu machines are running on cloud (DigitalOcen), I have to take the logs of those machines and ship those logs to Logstash where complete ELK is configured.
I have configured filebeat in one system and my filebeat.yml is like below:
when i start the logstash it is running successfully but I am not able to see any index in elasticsearch. I had tried multiple ways but no results can anyone help me Out of this.
And Is there any particular process is there for above scenario..
Please format your code/logs/config using the </> button, or markdown style back ticks. It helps to make things easy to read which helps us help you You may also want to redact your public IPs.
Do the filebeat logs show it connecting to Logstash?
I check my file beat logs /var/log/filebeat folder and the logs are
2020-03-10T03:28:07.717Z INFO instance/beat.go:468 Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]
2020-03-10T03:28:07.717Z INFO instance/beat.go:475 Beat UUID: 8a1fc2dc-b9a7-4d1b-ba72-9de650887499
Hi when i again start the filebeat i getting the error like below
<2020-03-10T05:33:22.567Z INFO pipeline/output.go:95 Connecting to backoff(async(tcp://206.189.129.234:5044))/>
<2020-03-10T05:33:31.545Z ERROR fileset/factory.go:105 Error creating input: Can only start an input when all related states are finished: {Id:262590-64513 Finished:false Fileinfo:0xc420518dd0 Source:/var/log/nginx/access.log Offset:0 Timestamp:2020-03-10 05:33:21.544219089 +0000 UTC m=+0.092800296 TTL:-1ns Type:log Meta:map FileStateOS:262590-64513} />
< 2020-03-10T05:33:31.548Z ERROR [reload] cfgfile/list.go:96 Error creating runner from config: Can only start an input when all related states are finished: {Id:262590-64513 Finished:false Fileinfo:0xc420518dd0 Source:/var/log/nginx/access.log Offset:0 Timestamp:2020-03-10 05:33:21.544219089 +0000 UTC m=+0.092800296 TTL:-1ns Type:log Meta:map FileStateOS:262590-64513} />
< 2020-03-10T05:33:41.553Z ERROR fileset/factory.go:105 Error creating input: Can only start an input when all related states are finished: {Id:262590-64513 Finished:false Fileinfo:0xc420518dd0 Source:/var/log/nginx/access.log Offset:0 Timestamp:2020-03-10 05:33:21.544219089 +0000 UTC m=+0.092800296 TTL:-1ns Type:log Meta:map FileStateOS:262590-64513} />
< 2020-03-10T05:33:41.556Z ERROR [reload] cfgfile/list.go:96 Error creating runner from config: Can only start an input when all related states are finished: {Id:262590-64513 Finished:false Fileinfo:0xc420518dd0 Source:/var/log/nginx/access.log Offset:0 Timestamp:2020-03-10 05:33:21.544219089 +0000 UTC m=+0.092800296 TTL:-1ns Type:log Meta:map FileStateOS:262590-64513} />
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.