Want to remove some field from GROK filter

Elastic Stack Logstash
1

I've this log format:

QID: 13636 from: 127.0.0.1 qtype: A qclass: IN qname: acep.top. rcode: NXDOMAIN rrcount: 0 policy: 'dns-name.host.sum'

I've used this GROK filter :
%{WORD:qid}: %{NUMBER:id} %{WORD:from}: %{IP:clientip} %{WORD:qtype}: %{WORD:qtans} %{WORD:qcls}: %{WORD:qclst} %{WORD:qname}: %{HOSTNAME:dstdom} %{WORD:rcode}: %{WORD:policyresp} %{WORD:rrcount}: %{NUMBER:rcnum} %{WORD:pol}: %{GREEDYDATA:policyzone}

I've this output:
{
"qid": [
[
"QID"
]
],
"id": [
[
"13636"
]
],
"BASE10NUM": [
[
"13636",
"0"
]
],
"from": [
[
"from"
]
],
"clientip": [
[
"127.0.0.1"
]
],
"IPV6": [
[
null
]
],
"IPV4": [
[
"127.0.0.1"
]
],
"qtype": [
[
"qtype"
]
],
"qtans": [
[
"A"
]
],
"qcls": [
[
"qclass"
]
],
"qclst": [
[
"IN"
]
],
"qname": [
[
"qname"
]
],
"dstdom": [
[
"acep.top."
]
],
"rcode": [
[
"rcode"
]
],
"policyresp": [
[
"NXDOMAIN"
]
],
"rrcount": [
[
"rrcount"
]
],
"rcnum": [
[
"0"
]
],
"pol": [
[
"policy"
]
],
"policyzone": [
[
"'dns-name.host.sum'"
]
]
}

I want to remove some fields like qcls , qclst from output.Can anyone help please.

2

You can remove fields from an event using mutate+remove_field. However, if you do not want to capture the value of a pattern then just do not name it, so instead of %{WORD:qcls}: just use %{WORD}:

That said, you appear to be using regular expression to match constants. It would be more efficient to put the constant in the pattern. For example if the message always starts with QID then start your pattern with

^QID: %{NUMBER:id}

You can read here about why anchoring that expression with ^ is a good idea.

1 Like
3

Many Thanks.

4

I want to discard the inverted comma from %{GREEDYDATA:policyzone}.

"policyzone": [
[
"'dns-name.host.sum'"
]
]

Now output coming with inverted comma,How I can remove that.

5 
mutate { gsub => [ "policyzone", "'", "" ] }
1 Like
6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.