Logstash 6.2.3 completely ignores configuration

Elastic Stack Logstash
1 
$ ./logstash --version
logstash 6.2.3


$ cat /etc/logstash/conf.d/logstash.conf
input {
	file {
		path => "/home/pkara/Desktop/mylogfile.log"
	}
}

output {
	stdout { }
}


$ cat /etc/logstash/pipelines.yml 
- pipeline.id: main
  path.config: "/etc/logstash/conf.d/*.conf"

However, when performing sudo journalctl -f -u logstash nothing appears in the stdout.

Neither in tail -500f /var/log/logstash/logstash-plain.log

What is more, when performing:

$ sudo ./logstash -f /etc/logstash/conf.d/
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[INFO ] 2018-03-22 17:59:12.505 [main] scaffold - Initializing module {:module_name=>"netflow", :directory=>"/usr/share/logstash/modules/netflow/configuration"}
[INFO ] 2018-03-22 17:59:12.514 [main] scaffold - Initializing module {:module_name=>"fb_apache", :directory=>"/usr/share/logstash/modules/fb_apache/configuration"}
[WARN ] 2018-03-22 17:59:13.102 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified
[INFO ] 2018-03-22 17:59:13.122 [LogStash::Runner] agent - No persistent UUID file found. Generating new UUID {:uuid=>"16d994a1-d125-4c30-9406-a307c1f8a4a6", :path=>"/usr/share/logstash/data/uuid"}
[INFO ] 2018-03-22 17:59:13.305 [LogStash::Runner] runner - Starting Logstash {"logstash.version"=>"6.2.3"}
[INFO ] 2018-03-22 17:59:13.457 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9600}
[INFO ] 2018-03-22 17:59:14.062 [Ruby-0-Thread-1: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:22] pipeline - Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[INFO ] 2018-03-22 17:59:14.693 [Ruby-0-Thread-1: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:22] pipeline - Pipeline started succesfully {:pipeline_id=>"main", :thread=>"#<Thread:0x72d8395f@/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:246 run>"}
[INFO ] 2018-03-22 17:59:14.743 [Ruby-0-Thread-1: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:22] agent - Pipelines running {:count=>1, :pipelines=>["main"]}

and the logs stay at this point forever!

Here is a gist with debug level info.

How is ever possible such a trivial thing to become so overly complicated to troubleshoot?

1 Like
2

If you want to parse a new file from the top you need start_position => "beginning" in your file input. In this case the file is no longer considered new so make sure you clear the sincedb file first. See the file input documentation for details.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.