Problem when adding fields to logstash

Elastic Stack Logstash
1

Hi guys! There was a problem adding the add_field via the mutate filter plugin. I do this:

        filter {
      if [type] == "syslog" {
        mutate {
          add_field => [ "received_at", "%{@timestamp}" ]
          add_field => [ "received_from", "%{host}" ]
        }
      }
    }

But fields not added to events. Where is my mistake?

All config:

input {
  beats {
    port => 5044
    type => "beats"
  }

  udp {
    port => 6514
    type => "syslog"
  }
  tcp {
    port => 6514
    type => "syslog"
  }
}


filter {

  if [type] == "syslog" {
    mutate {
        add_field => { "received_at" => "%{@timestamp}" }
    }
  }

}


  output {
    if [type] == "beats" {
      udp {
    #codec => plain { format => "%{message}"}
        codec => "json"
        host => "192.168.0.56"
        port => "6514"
      }
      }

    else if [type] == "syslog" {
            udp {
    codec => plain { format => "%{message}"}
    #    codec => "json"
    #    type => "syslog"
        host => "192.168.0.56"
        port => "514"
      }
      }

    }
2

What is the correct config you are using? On the first one your add_field is wrong, you are passing an array instead of a hash, but in your second config that you pasted it is right, you are passing a hash.

The correct format for add_field is this:

mutate {
    add_field => { "received_at" => "%{@timestamp}" }
}

Not this:

mutate {
    add_field => [ "received_at", "%{@timestamp}" ]
}

Which one are you using?

Can you share an example of the output of an event?

1 Like
3

I use

mutate {
    add_field => { "received_at" => "%{@timestamp}" }
}

Sure:

<86>Dec 5 10:10:56 ubuntu pkexec: pam_unix(polkit-1:session): session opened for user root by (uid=1000)

4

Looking into your full config, your output for when the field type has the value syslog is the one below.

udp {
    codec => plain { format => "%{message}"}
    host => "192.168.0.56"
    port => "514"
}

The line below is making you output only the original message in the field message, anything else will be ignored.

codec => plain { format => "%{message}"}

If you want to ouput the fields that you added, received_at and received_from you will need to add then to the format option.

codec => plain { format => "%{message} %{received_at} %{received_from"}
1 Like
5

Does not work :cold_sweat:
Config:

input {
  beats {
    port => 5044
    type => "beats"
  }

  udp {
    port => 8514
    type => "syslog"
  }
  tcp {
    port => 8514
    type => "syslog"
  }
}


filter {

  if [type] == "syslog" {
    mutate {
        add_field => { "received_at" => "%{@timestamp}" }
    }
  }

}


output{
if [type] == "beats" {
  udp {
#codec => plain { format => "%{message}"}
    codec => "json"
    host => "192.168.0.56"
    port => "8514"
  }
  }

else if [type] == "syslog" {
        udp {
codec => plain { format => "%{message} %{received_at}" }
#    codec => "json"
#    type => "syslog"
    host => "192.168.0.56"
    port => "514"
  }
  }

}

Output of an event:

<14>Dec 5 11:37:18 ubuntu NetworkManager[742]: [1607197038.8065] connectivity: (ens33) timed out

6

Sorry, it works. if you write in output codec: json, then the following fields are automatically added: timestump, host, version. How do I remove them? Use remove in filter?

UPD: problem solved, just needed some sleep =))

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.