Grokk Patterning output to respective indexes based on [agent.type] (or any conditionals) not working

Elastic Stack Beats
,
1

Hello,

Sorry but i've got a question as I have searched alot for an answer but could not find anything.
My winlogbeat and filebeat are all sent to logstash on 5044. I want logstash to filter using Grokk and output them to dedicated indexes. I am using the following config for logstash but it appears the condition if [agent][type] == "filebeat" is not working. Please tell me what have I got wrong, thanks!

Logstash Config

input {
beats {
port => 5044
}
}

filter {
if [agent][type] == "filebeat"{
if "404" in [message] {
grok {
match => {"message" => "%{TIMESTAMP_ISO8601:Grokkstamp} %{WORD:Grokkiis} %{WORD:Grokksite} %{IP:GrokkclientIP} %{WORD:Grokkhttp} %{GREEDYDATA:Grokkurl} %{NUMBER:Grokkport} %{NOTSPACE:Grokkusername} %{IPORHOST:Grokkclienthost} %{NOTSPACE:Grokkuseragent} %{NOTSPACE:Grokkreferer} %{DATA:Grokkhosturl} %{NUMBER:Grokkresponse} %{NUMBER:Grokksubresponse} %{NUMBER:Grokkscstatus} %{NUMBER:Grokkint}"}
}
}
else if "403" in [message] {
grok {
match => {"message" => "%{TIMESTAMP_ISO8601:Grokkstamp} %{WORD:Grokkiis} %{WORD:Grokksite} %{IP:GrokkclientIP} %{WORD:Grokkhttp} %{GREEDYDATA:Grokkurl} %{NUMBER:Grokkport} %{NOTSPACE:Grokkusername} %{IPORHOST:Grokkclienthost} %{NOTSPACE:Grokkuseragent} %{NOTSPACE:Grokkreferer} %{DATA:Grokkhosturl} %{NUMBER:Grokkresponse} %{NUMBER:Grokksubresponse} %{NUMBER:Grokkscstatus} %{NUMBER:Grokkint}"}
}
}
else if "400" in [message] {
grok {
match => {"message" => "%{TIMESTAMP_ISO8601:Grokkstamp} %{WORD:Grokkiis} %{WORD:Grokksite} %{IP:GrokkclientIP} %{WORD:Grokkhttp} %{GREEDYDATA:Grokkurl} %{NUMBER:Grokkport} %{NOTSPACE:Grokkusername} %{IPORHOST:Grokkclienthost} %{NOTSPACE:Grokkuseragent} %{NOTSPACE:Grokkreferer} %{DATA:Grokkhosturl} %{NUMBER:Grokkresponse} %{NUMBER:Grokksubresponse} %{NUMBER:Grokkscstatus} %{NUMBER:Grokkint}"}
}
}
else if "500" in [message] {
grok {
match => {"message" => "%{TIMESTAMP_ISO8601:Grokkstamp} %{WORD:Grokkiis} %{WORD:Grokksite} %{IP:GrokkclientIP} %{WORD:Grokkhttp} %{GREEDYDATA:Grokkurl} %{NUMBER:Grokkport} %{NOTSPACE:Grokkusername} %{IPORHOST:Grokkclienthost} %{NOTSPACE:Grokkuseragent} %{NOTSPACE:Grokkreferer} %{DATA:Grokkhosturl} %{NUMBER:Grokkresponse} %{NUMBER:Grokksubresponse} %{NUMBER:Grokkscstatus} %{NUMBER:Grokkint}"}
}
}
else {
}
}

else if [agent][type] == "winlogbeat"{
if "Service: THP" in [message] {
grok {
match => {"message" => "%{GREEDYDATA:message1} %{WORD:error}"}
}
}
else {
}
}
}

output
{
if [agent][type] == "filebeat"{
if "dd00-sap-iislogs" in [tags]{
elasticsearch
{
hosts => "${xxxx}"
user => "${xxxx}"
password => "${xxxx}"
index => "n1o0-rbptet-sre-windows-filebeat-%{+YYYY.MM}"
manage_template => false
ssl => true
ssl_certificate_verification => false
cacert => "/xxx_xxx/xxx/xxx-xxx.crt"
ilm_rollover_alias => "xxx-xxx-xxx-windows-filebeat"
ilm_pattern => "000001"
ilm_policy => "xxx-xxx-xxx-xxx"
}
}
}
else if [agent][type] == "winlogbeat"]{
elasticsearch
{
hosts => "${xxxx}"
user => "${xxxx}"
password => "${xxxx}"
index => "xxx-xxx-xxx-windows-winlogbeat-%{+YYYY.MM}"
manage_template => false
ssl => true
ssl_certificate_verification => false
cacert => "/xxx_xxx/xxx/xxx-xxx.crt"
ilm_rollover_alias => "xxx-xxx-xxx-windows-winlogbeat"
ilm_pattern => "000001"
ilm_policy => "xxx-xxx-xxx-xxx"
}
}
}

2

Hi @Rohan-boogeyman,

would it be possible for you to format your config using Preformated text instead of Block quote. It would be much easier to read :slight_smile:

Which version of Filebeat are you using? Can't see anything immediately wrong with the if [agent][type] == "filebeat" bit.

What is the result now? Is else if [agent][type] == "winlogbeat" working as expected?

1 Like
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.