Logstash 6.2.3 completely ignores configuration

$ ./logstash --version
logstash 6.2.3

$ cat /etc/logstash/conf.d/logstash.conf
input {
	file {
		path => "/home/pkara/Desktop/mylogfile.log"

output {
	stdout { }

$ cat /etc/logstash/pipelines.yml 
- pipeline.id: main
  path.config: "/etc/logstash/conf.d/*.conf"

However, when performing sudo journalctl -f -u logstash nothing appears in the stdout.

Neither in tail -500f /var/log/logstash/logstash-plain.log

What is more, when performing:

$ sudo ./logstash -f /etc/logstash/conf.d/
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[INFO ] 2018-03-22 17:59:12.505 [main] scaffold - Initializing module {:module_name=>"netflow", :directory=>"/usr/share/logstash/modules/netflow/configuration"}
[INFO ] 2018-03-22 17:59:12.514 [main] scaffold - Initializing module {:module_name=>"fb_apache", :directory=>"/usr/share/logstash/modules/fb_apache/configuration"}
[WARN ] 2018-03-22 17:59:13.102 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified
[INFO ] 2018-03-22 17:59:13.122 [LogStash::Runner] agent - No persistent UUID file found. Generating new UUID {:uuid=>"16d994a1-d125-4c30-9406-a307c1f8a4a6", :path=>"/usr/share/logstash/data/uuid"}
[INFO ] 2018-03-22 17:59:13.305 [LogStash::Runner] runner - Starting Logstash {"logstash.version"=>"6.2.3"}
[INFO ] 2018-03-22 17:59:13.457 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9600}
[INFO ] 2018-03-22 17:59:14.062 [Ruby-0-Thread-1: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:22] pipeline - Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[INFO ] 2018-03-22 17:59:14.693 [Ruby-0-Thread-1: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:22] pipeline - Pipeline started succesfully {:pipeline_id=>"main", :thread=>"#<Thread:0x72d8395f@/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:246 run>"}
[INFO ] 2018-03-22 17:59:14.743 [Ruby-0-Thread-1: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:22] agent - Pipelines running {:count=>1, :pipelines=>["main"]}

and the logs stay at this point forever!

Here is a gist with debug level info.

How is ever possible such a trivial thing to become so overly complicated to troubleshoot?

1 Like

If you want to parse a new file from the top you need start_position => "beginning" in your file input. In this case the file is no longer considered new so make sure you clear the sincedb file first. See the file input documentation for details.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.