Logstash errors out when running command for config

Elastic Stack Logstash
1

Hello and happy new year everyone.

I am running ELK on Ubuntu 16.04 and I have everything installed correctly it seems except for when I try to execute my commands and then thats when stuff goes down hill real quick.

The first command I try to run is
bin/logstash emails.conf
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
ERROR: Unknown command 'emails.conf'

_See: 'bin/logstash --help'_
_[ERROR] 2018-12-31 17:11:43.863 [main] Logstash - java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit_

I have also tried
bin/logstash -f emails.conf
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[FATAL] 2018-12-31 17:12:49.353 [main] runner - An unexpected error occurred! {:error=>#<ArgumentError: Path "/usr/share/logstash/data" must be a writable directory. It is not writable.>, :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/settings.rb:447:in validate'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:229:invalidate_value'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:140:in block in validate_all'", "org/jruby/RubyHash.java:1343:ineach'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:139:in validate_all'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:278:inexecute'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/clamp-0.6.5/lib/clamp/command.rb:67:in run'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:237:inrun'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/clamp-0.6.5/lib/clamp/command.rb:132:in run'", "/usr/share/logstash/lib/bootstrap/environment.rb:73:in'"]}
[ERROR] 2018-12-31 17:12:49.373 [main] Logstash - java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit

Then I tried
bin/logstash --path.settings emails.conf
ERROR: Failed to load settings file from "path.settings". Aborting... path.setting=emails.conf, exception=Java::JavaLang::RuntimeException, message=>Unhandled IOException: java.io.IOException: unhandled errno: Not a directory
[ERROR] 2018-12-31 17:14:36.602 [main] Logstash - java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit

Here is my config file I am running

input {
	file {
	path => "/home/xxxxxx/Documents/xxxxx/*.csv"
	start_position => "beginning"
	sincedb_path => "dev/null"
	}
}
	filter {
	csv {
	separator => ","
	}
}
output {
	elasticsearch {	host => ["http://localhost:9200"] }
	index => "xxxx"
}
stdout {codec => rubydebug}
}

Any help would be much appreciated.

2

So I ran

sudo bin/logstash -f /etc/logstash/conf.d/emails.conf --path.settings /etc/logstash

and it came back with another message so I guess I am making progress.

sudo bin/logstash -f /etc/logstash/conf.d/emails.conf --path.settings /etc/logstash
[sudo] password for dtengine: 
Sending Logstash logs to /var/log/logstash which is now configured via log4j2.properties
[2018-12-31T17:24:51,855][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2018-12-31T17:24:51,896][FATAL][logstash.runner          ] Logstash could not be started because there is already another instance using the configured data directory.  If you wish to run multiple instances, you must change the "path.data" setting.
[2018-12-31T17:24:51,914][ERROR][org.logstash.Logstash    ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit
3

I modified the Pipeline document and we seen to be in business.

I guess sometimes you gotta talk it out :slight_smile:

4

Hello you need to mention the column names as well like as shown below,

filter {
csv {
separator => ","
columns => ["open","high","low","close","volume"]
}
}

The sample example is as shown here.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.