Conditionnal filter not recognize field

Elastic Stack Logstash
1

Hi all,

I'm trying to use conditionnal filtering, with following config :
filebeat.yml

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/log/pure-ftpd.log
  fields:
    log_type: ftp_logs

output.logstash:
  # The Logstash hosts
  hosts: ["localhost:5044"]

logstash pipeline

input {
  beats {
    port => 5044
  }
}

 filter {
        if [log_type] == "ftp_logs"
                {
                grok    {
                        match => { "message" => "%{TIMESTAMP_ISO8601:host_timestamp} %{HOSTNAME:ftp_server} pure-ftpd: \((%{EMAILADDRESS:client_user}|%{USER:client_user}|\?)@%{HOSTNAME:client_ip}\) \[%{LOGLEVEL:loglevel}\] %{GREEDYDATA:message}" }
                overwrite => [ "message" ]
                        }

                geoip   {
                        source => "client_ip"
                        }
                }
}

output {
  stdout { codec => rubydebug { metadata => true } }
}

Example of input :

{
     "input" => {
    "type" => "log"
},
     "agent" => {
        "hostname" => "ftp2",
              "id" => "f1e8efca-109b-4559-b34f-8643a14b3763",
    "ephemeral_id" => "43950e78-9647-40ec-abe2-87b88487a164",
            "type" => "filebeat",
         "version" => "7.4.2"
},
"@timestamp" => 2019-11-28T16:14:02.721Z,
       "ecs" => {
    "version" => "1.1.0"
},
       "log" => {
      "file" => {
        "path" => "/var/log/pure-ftpd.log"
    },
    "offset" => 26338617
},
 "@metadata" => {
          "type" => "_doc",
          "beat" => "filebeat",
       "version" => "7.4.2",
    "ip_address" => "127.0.0.1"
},
  "@version" => "1",
      "host" => {
         "hostname" => "ftp2",
               "os" => {
          "kernel" => "4.15.18-21-pve",
        "codename" => "stretch",
            "name" => "Debian GNU/Linux",
          "family" => "debian",
         "version" => "9 (stretch)",
        "platform" => "debian"
    },
    "containerized" => false,
             "name" => "ftp2",
               "id" => "4aad43a5c5b149d7af767c43f4da41a0",
     "architecture" => "x86_64"
},
   "message" => "2019-11-28T17:13:05.898088+01:00 ftp2 pure-ftpd: (?@3.11.109.36) [INFO] panda is now logged in",
    "fields" => {
    "log_type" => "ftp_logs"
},
      "tags" => [
    [0] "beats_input_codec_plain_applied"
]
}

My grok filter works fine without the "if [log_type] == "ftp_logs".
I've tried other sy ntax [fields.log_type] == "ftp_logs" , and enen tried with other fields [type] = "log" with no success.

could you please telle me what is my mistake ?
thanks

thomas

2

You should reference that using

if [fields][log_type] == "ftp_logs" { ..
3

I hadn't found this syntax, now it works better indeed :slight_smile:
thanks @Badger !

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.