Hello there,
I'm a total newb to ELK, so at this point I am just trying to ingest a log file from a specific path using a logstash configuration file in conf.d . it's just not working, however...
This is what my conf file looks like:
file {
path => "/home/admin/Documents/secure"
start_position => beginning
}
}
filter {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:system.auth.timestamp} %{SYSLOGHOST:system.auth.hostname} sshd(?:\\[%{POSINT:system.auth.pid}\\])? %{DATA:system.auth.ssh.event} %{DATA:system.auth.sshmethod} for (invalid user )?%{DATA:system.auth.user} from %{IPORHOST:system.auth.ip} port %{NUMBER:system.auth.port} ssh2(: %{GREEDYDATA:system.auth.ssh.signature})? }
}
}
output {
elasticsearch {
hosts => "http://localhost:9200"
index => "securelog"
}
}
And this is what I get when I test it
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[INFO ] 2020-09-03 14:50:10.127 [main] runner - Starting Logstash {"logstash.version"=>"7.9.0", "jruby.version"=>"jruby 9.2.12.0 (2.5.7) 2020-07-01 db01a49ba6 OpenJDK 64-Bit Server VM 25.262-b10 on 1.8.0_262-b10 +indy +jit [linux-x86_64]"}
[WARN ] 2020-09-03 14:50:10.751 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified
[FATAL] 2020-09-03 14:50:11.562 [LogStash::Runner] runner - The given configuration is invalid. Reason: Expected one of [ \t\r\n], "#", "{", "}" at line 16, column 13 (byte 521) after filter {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:system.auth.timestamp} %{SYSLOGHOST:system.auth.hostname} sshd(?:\\[%{POSINT:system.auth.pid}\\])? %{DATA:system.auth.ssh.event} %{DATA:system.auth.sshmethod} for (invalid user )?%{DATA:system.auth.user} from %{IPORHOST:system.auth.ip} port %{NUMBER:system.auth.port} ssh2(: %{GREEDYDATA:system.auth.ssh.signature})? }
}
}
output {
elasticsearch {
hosts => "
[ERROR] 2020-09-03 14:50:11.566 [LogStash::Runner] Logstash - java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit
It seems like this should be super simple, but i just cant get it to work 
Any advice would be greatly appreciated!