Logstash filter fields in if conditions

Hello,

I am learning about logstash, esspecially about logstash filter with if condition. Something not clear to me is what are those fields used in if condition? How can I get the list of those fields?
For example, I want to apply different grok filter format for logs coming from different hosts. How can i put the condition to match hostname.

Your clarification here is highly appreciated!
Thanks!

There are many ways you could do. You can either do a if/else condition in output or within filter.
I tend to do mostly within filter as it will be cleaner and modular

So an example would be

input {
    pipeline {
        address => os_nix_syslog_pipeline
    }
}

filter {
  grok {
    match => {
        message => "%{SYSLOG5424LINE}"
      }
  }

  if "syslog5424_host" == '127.0.0.1' {
    mutate {
      add_field => { "myhost" => "localhost" }
    }  
  } else {
    mutate {
      add_field => { "myhost" => "unknown" }
    }  
  }
}

output {
  elasticsearch {
    hosts => "http://localhost:9200"
    user => "elastic"
    password => "changeme"
    index => "os_%{myhost}-%{+YYYY.MM}"
  }
}

The idea here is

  • Get the input from a pipeline or log. In this example, it is a linux_syslog
  • Grok to get the linux_syslog paramters
  • if the syslog_host is 127.0.0.1, its localhost and add a field called "myhost". If anything else, it is "unknown"
  • Just pump the output based on that field

So, you grok the event first to get the filed and then put the condition based on one of those fields, what if I want to grok based on the hostname from where the event comes, which field that I should use in my condition?
Something like this ?
if "database-server" in [hostname] {

}

really can't comment on that until I see the payload and fields you have already extracted.
Best thing to do is to see your raw data in Elastic, and then find the field (key-value) which has the "database-server". if it is beats, then it might be host.name or agent.hostname or beat.hostname depending on the version or type of input

So something like

if [host][name] == 'database-server' {

}
1 Like

Thanks a lot for your suggestion. Yes, I am using filebeat to ship postgres log to logstash.
After changing the condition, it works

if [host][hostname] == 'database-server' {

}

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.