Translate filter not mapping output

Elastic Stack Logstash
1

I have following test.conf file of logstash

input {
stdin {
codec => json
}
}

filter {
translate {
field => "user"
destination => "division"
dictionary_path => './PA_orga1_div.yml'

add_field => [ "div", "%{division}" ]

            }

    }

output {
stdout {
codec => plain {
charset => "ISO-8859-1"
}
}
}

The contents of PA_orga1_div.yml are
"sylvain" : "TRD_MEMS"
"arnaud" : "TRD_LAB"

On running the logstash with given configuration, I am not getting the "division" field in output corresponding to "sylvain"

../bin/logstash -f test.conf
Sending Logstash logs to /ELK/sw/logstash-6.4.2/logs which is now configured via log4j2.properties
[2019-04-17T16:24:46,451][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2019-04-17T16:24:46,987][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"6.4.2"}
[2019-04-17T16:24:49,091][INFO ][logstash.pipeline ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>10, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[2019-04-17T16:24:49,337][INFO ][logstash.inputs.stdin ] Automatically switching from json to json_lines codec {:plugin=>"stdin"}
[2019-04-17T16:24:49,408][INFO ][logstash.pipeline ] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0x29e58b57 run>"}
The stdin plugin is now waiting for input:
[2019-04-17T16:24:49,492][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>}
[2019-04-17T16:24:49,841][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}

{ "user" : "sylvain" }
2019-04-17T10:55:09.305Z dlhl2117 %{message}

2

That's working as expected. With a json_lines codec on the input (json gets replaced automatically with json_lines) there is no message field. And the default format for a plain output codec is to print the hostname followed by the message field.

If you replaced the plain codec with rubydebug you would see

{
  "division" => "TRD_MEMS",
      "user" => "sylvain",
[...]
       "div" => "TRD_MEMS",
"@timestamp" => 2019-04-17T12:29:38.648Z,
  "@version" => "1"
}
3

Thanks. It worked after changing output codec.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.