Logstash- /var/log/logstash - has no files

[gurumu (ng)]

Hello- I have installed Elasticsearch, Kibana and Logstash version 8.1.0 on my ubuntu VM. I am trying to ingest auth.log into ES by passing it through logstash. attached is the configuration file screenshots. I am not seeing the data being ingested and while troubleshooting that, I am not able to see the logstash-plain.log file at all. I am new to ELK stack and just following the documentation. Could you please advise where am I going wrong.

[ibra_013 (Ibra13)]

Hi,

could you add this line to your logstash config,

output {
  elasticsearch { hosts => ["localhost:9200"] }
  stdout { codec => rubydebug }
}

and please run this command on the cli and see what happens

bin/logstash -f  /etc/logstash/conf.d/authlog.conf

[gurumu (ng)]

ng@ubuntu:/usr/share/logstash$ bin/logstash -f  /etc/logstash/conf.d/authlog.conf

OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.headius.backport9.modules.Modules (file:/usr/share/logstash/logstash-core/lib/jars/jruby-complete-9.2.8.0.jar) to field java.io.FileDescriptor.fd
WARNING: Please consider reporting this to the maintainers of com.headius.backport9.modules.Modules
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Thread.exclusive is deprecated, use Thread::Mutex
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[FATAL] 2022-05-03 12:28:19.705 [main] runner - An unexpected error occurred! {:error=>#<ArgumentError: Path "/usr/share/logstash/data" must be a writable directory. It is not writable.>, :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/settings.rb:489:in `validate'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:271:in `validate_value'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:182:in `block in validate_all'", "org/jruby/RubyHash.java:1417:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:181:in `validate_all'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:283:in `execute'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/clamp-0.6.5/lib/clamp/command.rb:67:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:242:in `run'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/clamp-0.6.5/lib/clamp/command.rb:132:in `run'", "/usr/share/logstash/lib/bootstrap/environment.rb:73:in `<main>'"]}
[ERROR] 2022-05-03 12:28:19.734 [main] Logstash - java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit

[ibra_013 (Ibra13)]

Hi,

sorry i did not precise where to run the command, it should be on /etc/logstash/

[gurumu (ng)]

ng@ubuntu:/etc/logstash$ bin/logstash -f /etc/logstash/conf.d/authlog.conf
-bash: bin/logstash: No such file or directory
ng@ubuntu:/etc/logstash$

[gurumu (ng)]

hi ibra - am I missing something here? when I run the command, it says no such file or directory

[ibra_013 (Ibra13)]

Hi,

let do this again

ng@ubuntu:/etc/logstash# /usr/share/logstash/bin/logstash  -f  /etc/logstash/conf.d/authlog.conf 

[gurumu (ng)]

hi ibra - It worked after elevating the privileges to root.

root@ubuntu:/usr/share/logstash# cd /etc/logstash
root@ubuntu:/etc/logstash# /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/authlog.conf

thank you so much.