Remove a value in grok filter

Elastic Stack Logstash
1

Hi All,

I have Prepared Logstash configuration code

filter {
multiline {
pattern => "^####"
negate => true
what => "previous"
}

grok {
pattern => ["####<%{DATA:Timestamp}> <%{WORD:Severity}> %{GREEDYDATA:LogMessage}"]

}

To analyse log file content like below.

####<Jul 26, 2015 12:02:00 AM EDT> <eag_dfg> <[ACTIVE] ExecuteThread: '14' for queue: 'weblogic.kernel.Default (self-tuning)'> <> a80c994595c9e109:-2a0044f6:14ea45c3fe1:-8000-000000000000ae03 <1437883320375>
####<Jul 26, 2015 12:02:00 AM EDT> <eag_dfg> <[ACTIVE] ExecuteThread: '14' for queue: 'weblogic.kernel.Default (self-tuning)'> <> a80c994595c9e109:-2a0044f6:14ea45c3fe1:-8000-000000000000ae03 <1437883320409>
####<Jul 26, 2015 12:02:00 AM EDT> <eag_dfg> <DynamicListenThread[Default[2]]> <> <> a80c994595c9e109:-2a0044f6:14ea45c3fe1:-8000-0000000000000006 <1437883320416> <Channel "Default[2]" listening on 127.0.0.1:7045 was shutdown.>

It is working as excepted.

I prepared a filter code like, 2nd field in log lines is storing in Severity column of Logstash.

Now i'm trying to remove log lines, when ever the value for Severity= Info.

I tried below. But it didn't work.

ruby {
code => "event.to_hash.delete_if {|Severity, value| value == 'Info' }"
}

Please assist me.

2 
filter {
  if [Severity] == "Info" {
    drop { }
  }
}
1 Like
3

Thank you. That helped me.

Now similarly i have been trying to remove

LogMessage Filed contains data like :

Tried with below code . It didn't work.

if [LogMessage] == ""
{
drop { }
}

Could you please assist me on this.

Regards,
Bharath

4

Above reply Missed some data..

Now similarly i have been trying to remove

LogMessage Filed contains data like : Enququeing to Error hospital...
Enququeing to Error hospital successful...

Tried with below code . It didn't work.

if [LogMessage] == "Enququeing to Error hospital successful..."
{
drop { }
}

Could you please assist me on this.

Regards,
Bharath

5

(I'm pretty sure you can edit old posts if you need to correct something.)

There's no structural problem with that configuration snippet. I suspect the LogMessage field simply isn't equal to exactly "Enququeing to Error hospital successful...".

If you can supply a minimal example that exhibits the problem it would be easier to help.

6

Thank you.

I made changes in main pattern filter. Then i tried the code. It worked.

Thanks for your valuable suggestion.

7

Hi,

I'm trying to remove LogMessage field contains output like

[43144] Skipping execution of Initialization Block: 'LAST_SYND_DS_YTD_QTD'
[43144] Skipping execution of Initialization Block: 'LAST_SYND_IDS_YTD_QTD'
[43144] Skipping execution of Initialization Block: 'LAST_SYND_RX_YTD_QTD'

All lines contains [43144] as common. So tried below to remove those lines. But it didn't work.

if [LogMessage] in [43144]
{
drop { }
}

Is there a way to compare part of string in if condition ?

Please assist.

Regards,
Bharath'

8 
if "[43144]" in [LogMessage] {
  drop { }
}
9

Thank you.

It worked.