Hello,
I try to analyse loggimg output of iptables, which can be grokked by logstash with kv filter.
For example
"IN" => "eth0",
"SRC" => "10.10.10.10",
"DST" => "192.168.100.100",
"PROTO" => "TCP",
"SPT" => "43852",
"DPT" => "22",
How can I make these fields analysable with kibana / elasticsearch?
Appendix:
My setup
filebeat
-
paths:
- /var/log/iptables.log
input_type: log
document_type: netfilter
logstash 2.2.2:
input {
beats { port => 5044 }
}
filter {
if [type] == "netfilter" {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:[%{POSINT:syslog_pid}])?: %{GREEDYDATA:syslog_message}" }
}
kv {
source => "syslog_message"
include_keys => [ "IN" , "OUT" , "SRC" , "DST" , "PROTO" , "SPT" , "DPT" ]
}
geoip {
source => "SRC"
database => "/etc/logstash/GeoLiteCity.dat"
}
date {
locale => "en"
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
}
output {
stdout { codec => rubydebug }
elasticsearch {
hosts => "localhost:9200"
}
}
elasticsearch 2.2.
The properties of mapping I can query but I don't know how to change it . DPT,DST,SRC,SPT.. for each index separatly or can I define an global behavior?
curl -s -XGET 'http://localhost:9200/logstash-2016.03.17' | jq '.[] | .mappings.netfilter.properties'
"DPT": {
"type": "string",
"norms": {
"enabled": false
},
"fielddata": {
"format": "disabled"
},
"fields": {
"raw": {
"type": "string",
"index": "not_analyzed",
"doc_values": true,
"ignore_above": 256
}
}
},
I found a lot of explanations in the web , but I didn't find a consistent example . Perhaps sombody can help me.
Thank you