Mutate - replace source value

Elastic Stack Logstash
1

I am trying to change the value of the @source field in my syslogs using mutate - replace. logstash configuration:

[/opt/logstash/bin] # cat /etc/logstash/conf.d/logstash.conf
input {

tcp {
port => 5000
type => syslog }

udp {
port => 5000
type => syslog }

gelf {
port => 12201
type => windows
codec => "json" }

}

filter {

if [message] =~ /default send string/ {
drop {}
}

if [type] == "syslog" {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
add_field => [ "received_at", "%{@timestamp}" ]
add_field => [ "received_from", "%{host}" ]
}

syslog_pri { }
date { match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ] }

}

if [source] == "0\:0\:0\:0\:0\:0\:0\:1" {
mutate {
  replace => [ "@source", "{%syslog_hostname}" ]
  }
}

date {
match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ]
}

}

output {

gelf {
   host => "xxxxxxxxx"
   port => 12202
 }

}

What I am trying to do is if the value of source field is 0:0:0:0:0:0:0:1 change it to the value of syslog_hostname field.

The logs are still showing 0:0:0:0:0:0:0:1 as the source value.

I have also tried 0:0:0:0:0:0:0:1 instead of 0:0:0:0:0:0:0:1 and if [@source] instead of if [source] in the configuration file.

Thank you in advance.

2

There are a couple of problems with this configuration:

  • Is the field named source or @source? Either way you're referring to the field inconsistently (sometimes source and sometimes @source).
  • The colon shouldn't be escaped, so your conditional should say if [source] == "0:0:0:0:0:0:0:1" { (assuming the field is named source).
  • {%syslog_hostname} should be %{syslog_hostname}.
3

Thank you!

I have corrected the config as follows:

if [source] == "0:0:0:0:0:0:0:1" {
mutate {
  replace => [ "source", "%{syslog_hostname}" ]
  }
}

But logs are still coming into elasticsearch with 0:0:0:0:0:0:0:1 value in the source field.

4

Reduce your example to the bare minimum so that you can debug it more easily. Use stdin and stdout for input/output and have nothing but your mutate filter. Like this:

input { stdin { codec => json } }
output { stdout { codec => rubydebug } }
filter {
  if [source] == "0:0:0:0:0:0:0:1" {
  mutate {
    replace => [ "source", "%{syslog_hostname}" ]
    }
  }
}

Then feed this on stdin of a Logstash process with the configuration above:

{"source": "0:0:0:0:0:0:0:1", "syslog_hostname": "foo"}

Do you get the expected results?

5

It works now, it was my mistake :frowning: Thank you for your help!

6

Hi I am trying to replace field "component " but not showing in output what exactly i am doing wrong . I am new to ELK. below is my logstash config.

input {
tcp {
'port' => '9563'
}
}

filter {
json {
source => 'message'
}
if [component] == "0" {
mutate {
replace => [ "component", "others" ]
}
}
if [component] == "1" {
mutate {
replace => [ "component", "datawarehouse" ]
}
}
}

output {
stdout { codec => rubydebug }
elasticsearch{ hosts => "90.0.3.63:9200" }
file {
'path' => '/tmp/output.log'
}
}

7

This thread is quite old. Please start your own thread instead.

8

i too face the same scenario..someone please guide us,Thank you