Logstash doesn't read an input file :/

Elastic Stack Logstash
1

Hello ,
I am trying to use Docker ELK stack to visualize the log of openvswitch.
My problem with logstash that it doesn't read the log file.
in the /logstash/pipeline/logstash.conf I added

input {
file {
    path => "/var/log/openvswitch/ovs-ctl.log"
    tags => "openvswitch"
    type => "openvswitch"
} 
}

when I am running the stack It doesn't read that file. I tried with another simple file in different location and also It doesn't read that file.
I have:

logstash_1       | Sending Logstash's logs to /usr/share/logstash/logs which is now configured via log4j2.properties
logstash_1       | [2017-07-31T18:07:52,990][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://elasticsearch:9200/]}}
logstash_1       | [2017-07-31T18:07:52,997][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://elasticsearch:9200/, :path=>"/"}
logstash_1       | [2017-07-31T18:07:53,187][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>#<URI::HTTP:0x505b9cd URL:http://elasticsearch:9200/>}
logstash_1       | [2017-07-31T18:07:53,190][INFO ][logstash.outputs.elasticsearch] Using mapping template from {:path=>nil}
logstash_1       | [2017-07-31T18:07:53,310][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"template"=>"logstash-*", "version"=>50001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"_all"=>{"enabled"=>true, "norms"=>false}, "dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword"}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date", "include_in_all"=>false}, "@version"=>{"type"=>"keyword", "include_in_all"=>false}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}}
logstash_1       | [2017-07-31T18:07:53,324][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>[#<URI::Generic:0x33286cfc URL://elasticsearch:9200>]}
logstash_1       | [2017-07-31T18:07:53,334][INFO ][logstash.pipeline        ] Starting pipeline {"id"=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>500}
logstash_1       | [2017-07-31T18:07:53,828][INFO ][logstash.inputs.tcp      ] Starting tcp input listener {:address=>"0.0.0.0:5000"}
logstash_1       | [2017-07-31T18:07:53,849][INFO ][logstash.pipeline        ] Pipeline main started
logstash_1       | [2017-07-31T18:07:54,041][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}

Thank you in advance,

2

Can you attach your configuration file?

3

Does /var/log/openvswitch/ovs-ctl.log exist inside the Logstash Docker container?

4 
input {
	tcp {
		port => 5000
	}

	 file {
 	    path => "/var/log/openvswitch/ovs-ctl.log"
	    tags => "openvswitch"
	    type => "openvswitch"
	}
 }     

 output {
	elasticsearch {
		hosts => "elasticsearch:9200"
	}
}
5

No... but It's on the same device. Why and how it should be in the Logstash Docker container?

6

I had the same issue with running Logstash using Docker. The container's files are separate from the files on the device unless you mount a volume containing the files you wish to parse. Basically logstash is looking within the containers file system to find the file you are specifying. It cannot find that file because it does not exist within that container. Read more about how to create a container here.

7

Many Thanks @CDR !!! Problem solved

You are right... I mounted the file to a "mediator" volume in the container (I used /usr/share/logstash/docs) and I modified the path of the input file to the mediator path.

Cheers!

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.