Logstash mutate split


(Man Kyun Choi) #1

I installed ELK and try to configured logstash to parse our Web Application FIrewall security log.

The WAF log is below

May 31 13:47:11 2017 WAFK_TAC (warning) log_processor: [WEBFRONT/0x00727001] Violated Cross Site Scripting - the form field is not allowed. (log_id="1882409510",app_name="default_app",app_id="255",src_if="lan",src_ip="192.168.204.139",src_port="58286",dest_ip="125.209.214.100",dest_port="80",forwarded_for="",host="ts.naver.com",url="/t",sig_warning="user",url_param="",protocol="http",block="no",evidence_id="1879446843",owasp="A3",field="bi", sigid="210700002" , data="\r\n")

I want to divide this message like this :

log_id="1882409510"
app_name="default_app"
app_id="255"
.
.
.

my logstash.conf is below

input {
file {
path => "/var/log/syslog"
start_position => "beginning"
}
}

filter {
if [path] == "syslog" {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:[%{POSINT:syslog_pid}])?: %{GREEDYDATA:syslog_message}" }
add_field => [ "received_at", "%{@timestamp}" ]
add_field => [ "received_from", "%{host}" ]
}

mutate {
split => {"syslog_message" => "="}
add_field => { "log_id" => "%{[syslog_message][0]}"
"app_name" => "%{[syslog_message][1]}"}
}

date {
  match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
}

}
}

output {
elasticsearch { hosts => ["127.0.0.1:9200"] }
stdout { codec => rubydebug }

===============================================

But, I can't see new filed.
I think there is something wrong but I can't find it
please help me


(Guy Boertje) #2

look at the KV filter or the dissect filter.


(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.