Logstash kv{} issue with field without any value


(Rolf Berkenbosch) #1

Hi there,

I have the following input:
<134> device="SFW" date=2016-11-15 time=00:00:35 timezone="CET" device_name="SG430" device_id=S50049065BB50F7 log_id=010302602002 log_type="Firewall" log_component="Appliance Access" log_subtype="Denied" status="Deny" priority=Information duration=0 fw_rule_id=0 policy_type=0 user_name="" user_gp="" iap=0 ips_policy_id=0 appfilter_policy_id=0 application="" application_risk=0 application_technology="" application_category="" in_interface="PortMGMT" out_interface="" src_mac=78:ac:c0:88:a8:aa src_ip=0.0.0.0 src_country_code= dst_ip=255.255.255.255 dst_country_code= protocol="UDP" src_port=68 dst_port=67 sent_pkts=0 recv_pkts=0 sent_bytes=0 recv_bytes=0 tran_src_ip= tran_src_port=0 tran_dst_ip= tran_dst_port=0 srczonetype="" srczone="" dstzonetype="" dstzone="" dir_disp="" connid="" vconnid="" hb_health="No Heartbeat"

But logstash index it as follows:

sent_pkts 0
src_country_code dst_ip=255.255.255.255
src_ip 0.0.0.0

How can I prevent logstash to index key without values. Is there a replacement string that I can run in front. If I find the following: '= ' replace it with '=""' before kv{} does it job.


(Mark Walkom) #2

What does your config look like?
What version?


(Rolf Berkenbosch) #3

I userd logstash-5.0.0-1

The config is as follows:

input {
tcp {
host => "172.39.39.111"
port => 5000
type => syslog
}
udp {
port => 5000
type => syslog
}
}

filter {
if [type] == "syslog" {
kv {
field_split => " "
include_brackets => false
}
date {
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
geoip {
source => "dst_ip"
}
}
}

output {
elasticsearch { hosts => ["localhost:9200"] }
stdout { codec => rubydebug }
}


(Rolf Berkenbosch) #4

I find the solutoin, place this in front of KV {}:

mutate {
gsub => [
"message", "= ", '="" '
]
}


(system) #5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.