*error message :
--path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[WARN ] 2020-05-20 22:06:21.916 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified
[INFO ] 2020-05-20 22:06:21.935 [LogStash::Runner] runner - Starting Logstash {"logstash.version"=>"7.7.0"}
[ERROR] 2020-05-20 22:06:24.665 [Converge PipelineAction::Create] agent - Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [ \t\r\n], "#", "=>" at line 8, column 8 (byte 118) after input {\n beats {\n port => 5044\n host => "192.168.200.167"\n sincedb_path => "/dev/null"\n }\nfilter {\n grok ", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:58:in compile_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:66:in compile_graph'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:28:in block in compile_sources'", "org/jruby/RubyArray.java:2577:in map'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:27:in compile_sources'", "org/logstash/execution/AbstractPipelineExt.java:181:in initialize'", "org/logstash/execution/JavaBasePipelineExt.java:67:in initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:43:in initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:52:in execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:342:in block in converge_state'"]}
[INFO ] 2020-05-20 22:06:25.279 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9600}
[INFO ] 2020-05-20 22:06:30.139 [LogStash::Runner] runner - Logstash shut down.
*log example:
2020-05-19T02:59:56.062564+09:00 127.0.0.1 [IPS-7514] [Attack_Name=SQL Injection_1], [Time=2020/05/19 02:59:49], [Hacker=0.0.0.0], [Victim=0.0.0.0], [Protocol=icmp/2048], [Risk=High], [Handling=Defence], [Information=], [SrcPort=0], [HackType=04100]
*grok pattern:
%{IP:host_ip}\s+[%{DATA:host}]\s+[Attack_Name=%{DATA:attack_name}],\s+[Time=%{DATA:attack_time}],\s+[Hacker=%{IP:src_ip}],\s+[Victim=%{IP:dst_ip}],\s+[Protocol=%{DATA:protocol}],\s+[Risk=%{WORD:risk}],\s+[Handling=%{DATA:handling}],\s+[Information=%{DATA:info}],\s+[SrcPort=%{INT:src_port}],\s+[HackType=%{DATA:hack_type}]
*config file :
input {
beats {
port => 5044
host => "192.168.200.167"
sincedb_path => "/dev/null"
}
filter {
grok {
match => {"message" => "%{IP:host_ip},\s+[%{DATA:host}],\s+[Attack_Name=%{DATA:attack_name}],\s+[Time=%{DATA:attack_time}],\s+[Hacker=%{IP:src_ip}],\s+[Victim=%{IP:dst_ip}],\s+[Protocol=%{DATA:protocol}],\s+[Risk=%{WORD:risk}],\s+[Handling=%{DATA:handling}],\s+[Information=%{DATA:info}],\s+[SrcPort=%{INT:src_port}],\s+[HackType=%{DATA:hack_type}]"}
}
date{ match => ["attack_time"= "YYYY/MM/DD HH:mm:SS"]
target => "@timestamp"}
}
output {
elasticsearch {
hosts => "localhost:9200"
manage_template => false
index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
}
}
OS = centos 7
logstash version = 7.7
es, kibana ver. = 7.7
