Hi all,
i getting error, while to start logstash.
logstash log is as bellow.
[2023-02-27T15:18:50,526][FATAL][org.logstash.Logstash ] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:747) ~[jruby-complete-9.2.20.1.jar:?]
at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:710) ~[jruby-complete-9.2.20.1.jar:?]
at usr.share.logstash.lib.bootstrap.environment.(/usr/share/logstash/lib/bootstrap/environment.rb:94) ~[?:?]
[2023-02-27T15:19:11,875][INFO ][logstash.runner ] Log4j configuration path used is: /etc/logstash/log4j2.properties
[2023-02-27T15:19:11,892][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"7.17.9", "jruby.version"=>"jruby 9.2.20.1 (2.5.8) 2021-11-30 2a2962fbd1 OpenJDK 64-Bit Server VM 11.0.18+10 on 11.0.18+10 +indy +jit [linux-x86_64]"}
[2023-02-27T15:19:11,896][INFO ][logstash.runner ] JVM bootstrap flags: [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djdk.io.File.enableADS=true, -Djruby.compile.invokedynamic=true, -Djruby.jit.threshold=0, -Djruby.regexp.interruptible=true, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true]
[2023-02-27T15:19:13,920][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false}
[2023-02-27T15:19:15,619][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [ \t\r\n], "#", "{", "}" at line 4, column 47 (byte 93) after filter {\n if [type] == "syslog" {\n grok {\n match => { "message" => "\ device_name="", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:32:in compile_imperative'", "org/logstash/execution/AbstractPipelineExt.java:189:in initialize'", "org/logstash/execution/JavaBasePipelineExt.java:72:in initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:48:in initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:52:in execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:392:in block in converge_state'"]}
[2023-02-27T15:19:15,769][INFO ][logstash.runner ] Logstash shut down.
[2023-02-27T15:19:15,785][FATAL][org.logstash.Logstash ] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:747) ~[jruby-complete-9.2.20.1.jar:?]
at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:710) ~[jruby-complete-9.2.20.1.jar:?]
at usr.share.logstash.lib.bootstrap.environment.(/usr/share/logstash/lib/bootstrap/environment.rb:94) ~[?:?]
and my logstash conf file image uploaded.Please find the attachment.
syslog-filter.conf file as below.
filter {
if [type] == "syslog" {
grok {
match => { "message" => "\ device_name="%{WORD:device_name}" timestamp="%{TIMESTAMP_ISO8601:timestamp}" device_model="%{WORD:device_model}" device_serial_id="%{WORD:device_serial_id}" log_id="%{DATA:log_id}" log_type="%{WORD:log_type}" log_component="%{DATA:ignored}" log_subtype="%{WORD:log_subtype}" log_version=%{INT:log_version} severity="%{WORD:ignored}" fw_rule_id="%{DATA:ignored}" nat_rule_id="%{DATA:ignored}" fw_rule_type="%{DATA:ignored}" ether_type="%{DATA:ignored}" src_ip="%{IP:src_ip}" src_country="%{WORD:src_country}" dst_ip="%{IP:dst_ip}" dst_country="%{WORD:dst_country}" protocol="%{DATA:ignored}" src_port=%{INT:src_port} dst_port=%{INT:dst_port} hb_status="%{DATA:ignored}" %{GREEDYDATA:message}"" }
add_field => [ "received_at", "%{@timestamp}" ]
add_field => [ "received_from", "%{host}" ]
}
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:[%{POSINT:syslog_pid}])?: %{GREEDYDATA:syslog_message}" }
add_field => [ "received_at", "%{@timestamp}" ]
add_field => [ "received_from", "%{host}" ]
}
date {
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
}
my syslem log : _gateway device_name="SFW" timestamp="2023-02-27T14:06:26+0530" device_model="XG310" device_serial_id="C320AB9KCCKVQEA" log_id="010202601001" log_type="Firewall" log_component="Invalid Traffic" log_subtype="Denied" log_version=1 severity="Information" fw_rule_id="N/A" nat_rule_id="0" fw_rule_type="NETWORK" ether_type="IPv4 (0x0800)" src_ip="10.100.12.100" src_country="R1" dst_ip="34.120.208.123" dst_country="USA" protocol="TCP" src_port=50780 dst_port=443 hb_status="No Heartbeat" message="Could not associate packet to any connection." app_resolved_by="Signature" app_is_cloud="FALSE" qualifier="New"
Please guide my how can I parse this log files.