I edited the logstash config file and now logstash does not work.
The following error occurs in logstash-plain.log.
# tail /var/log/logstash/logstash-plain.log
at usr.share.logstash.lib.bootstrap.environment.<main>(/usr/share/logstash/lib/bootstrap/environment.rb:89) ~[?:?]
[2021-04-06T14:52:18,341][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"7.11.0", "jruby.version"=>"jruby 9.2.13.0 (2.5.7) 2020-08-03 9a89c94bcc OpenJDK 64-Bit Server VM 11.0.8+10 on 11.0.8+10 +indy +jit [linux-x86_64]"}
[2021-04-06T14:52:18,830][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2021-04-06T14:52:19,409][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [ \\t\\r\\n], \"#\", \"input\", \"filter\", \"output\" at line 9, column 1 (byte 290) after ", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:32:in `compile_imperative'", "org/logstash/execution/AbstractPipelineExt.java:184:in `initialize'", "org/logstash/execution/JavaBasePipelineExt.java:69:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:47:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:52:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:367:in `block in converge_state'"]}
[2021-04-06T14:52:19,602][INFO ][logstash.runner ] Logstash shut down.
[2021-04-06T14:52:19,612][FATAL][org.logstash.Logstash ] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:747) ~[jruby-complete-9.2.13.0.jar:?]
at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:710) ~[jruby-complete-9.2.13.0.jar:?]
at usr.share.logstash.lib.bootstrap.environment.<main>(/usr/share/logstash/lib/bootstrap/environment.rb:89) ~[?:?]
However, there is no error in the edited config file (on line 9, of course), and execution with the --config.test_and_exit option is OK.
# /usr/share/logstash/bin/logstash -f /etc/logstash/first-pipeline.conf --config.test_and_exit
... snip ...
Configuration OK
[INFO ] 2021-04-06 14:52:05.305 [LogStash::Runner] runner - Using config.test_and_exit mode. Config Validation Result: OK. Exiting Logstash
Where is the error in this config?
# The # character at the beginning of a line indicates a comment.
# Use comments to describe your configuration.
input {
beats {
port => "5044"
}
}
# The filter part of this file is commented out to indicate that
# it is optional.
filter {
if ( [tags][0] == "secure" ) {
# see. https://mebee.info/2020/07/13/post-11525/
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:timestamp}\s+%{IPORHOST:dst_host}\s+%{WORD:syslog_program}\[\d+\]:\s+(?<status>.+)\s+for\s+%{USER:auth_user}\s+from\s+%{SYSLOGHOST:src_host}.*" }
add_field => { "activity" => "SSH Logins" }
add_tag => "linux_auth"
}
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:timestamp}\s+%{IPORHOST:dst_host}\s+%{WORD:syslog_program}\[\d+\]:\s+(?<status>.+)\s+for\s+invalid\s+user\s%{USER:auth_user_nonexist}\s+from\s+%{SYSLOGHOST:src_host}.*" }
add_field => { "activity" => "SSH Logins" }
add_tag => "linux_auth"
}
}
###
else if ( [tags][0] == "cas" or "cas" in [log][file][path] ) {
dissect {
mapping => {
"message" => "%{idate} %{itime} [%{ithread-name}] %{ilog-level} %{iclass-name} - %{ilog-message}"
}
}
mutate {
add_field => { "timestamp" => "%{idate} %{itime}" }
add_field => { "thread-name" => "%{ithread-name}" }
add_field => { "log-level" => "%{ilog-level}" }
add_field => { "class-name" => "%{iclass-name}" }
add_field => { "log-message" => "%{ilog-message}" }
}
}
###
}
output {
if( [tags][0] == "messages" or "messages" in [log][file][path] ) {
elasticsearch {
hosts => ["localhost"]
index => "messages-%{+YYYY.MM.dd}"
}
}
else if( [tags][0] == "nginx" or "nginx" in [log][file][path] ) {
if( [tags][1] == "access" or "access" in [log][file][path] ) {
elasticsearch {
hosts => ["localhost"]
index => "nginx-access-%{+YYYY.MM.dd}"
}
}
else if( [tags][1] == "error" or "error" in [log][file][path] ) {
elasticsearch {
hosts => ["localhost"]
index => "nginx-error-%{+YYYY.MM.dd}"
}
}
}
else if( [tags][0] == "elasticsearch" or "elasticsearch" in [log][file][path] ){
elasticsearch {
hosts => ["localhost"]
index => "elasticsearch-%{+YYYY.MM.dd}"
}
}
else if( [tags][0] == "secure" or "secure" in [log][file][path] ) {
elasticsearch {
hosts => ["localhost"]
index => "secure-%{+YYYY.MM.dd}"
}
}
###
else if( [tags][0] == "cas" or "cas" in [log][file][path] ) {
if ( [tags][1] == "server" or "server" in [log][file][path] ) {
elasticsearch {
hosts => ["localhost"]
index => "cas-server-%{+YYYY.MM.dd}"
}
}
else if ( [tags][1] == "error" or "error" in [log][file][path] ) {
elasticsearch {
hosts => ["localhost"]
index => "cas-error-%{+YYYY.MM.dd}"
}
}
}
else if( [tags][0] == "api" or "api" in [log][file][path] ) {
elasticsearch {
hosts => ["localhost"]
index => "api-%{+YYYY.MM.dd}"
}
}
###
else {
elasticsearch {
hosts => ["localhost"]
index => "other-%{+YYYY.MM.dd}"
}
}
}