Some problem of if and mutate in logstash filter

[zxc654951]

I had json format log and outputto elasticseach. I try to use "if" and "mutate" which copy Message to new field. But the new field can't viewed in elasticsearch and kibana. My Message field like "S", "E", "any string"...
and my code like below.

filter{
if [Message]=="S"{
mutate{copy=>{"Message"=>"action"}}
}
else if [Message]=="E"{
mutate{copy=>{"Message"=>"action"}}
}
}

Another try：

filter{
if [Message]=="S"{
add_field{"Message"=>"Action"}
mutate{copy=>{"Message"=>"action"}}
}
else if [Message]=="E"{
mutate{copy=>{"Message"=>"action"}}
}
}

Thanks for answer.

[magnusbaeck (Magnus Bäck)]

Please show an example event produced by Logstash. You can use a stdout { codec => rubydebug } output to dump a raw representation of the event.

[zxc654951]

filter{
date{
match=>["LogDateTime", "yyyy-MM-dd HH:mm:ss.SSSSSS"]
target=>"LogDateTime_2"
}
if [Message]=="S"{
mutate{
add_field=>{"ActionState"=>"Message"}
copy=>{"Message" => "ActionState"}}
}
}

Events like below：

{
"NhiSystemID" => " AAXX1001X01",
"LogDateTime_2" => 2018-04-30T08:53:38.626Z,
"LogDateTime" => "2018-04-30 16:53:38.626668",
"Message" => "S",
"LogLevel" => "info",
"TransactionID" => "2a693137 ",
"path" => "/home/tim/log test/20180430_2.log",
"@timestamp" => 2018-05-07T01:47:56.639Z,
"Scope" => "GEMFIRE",
"Daemon" => "4",
"@version" => "1",
"host" => "tim-VirtualBox",
"NhiActionID" => "11",
"ProcessID" => "0x00008X7X",
"ActionState" => [
[0] "S",
[1] "Message"
]
}

[magnusbaeck (Magnus Bäck)]

Okay, and what's the problem?

Keep in mind that the options given to the mutate filter aren't necessarily executed in the order given. If you have different mutate operations that depend on each other you should use different mutate filters. In this particular case the copy operation will run first, followed by the add_field operation.

[zxc654951]

Thanks, it work by "add_field" before "if" and "copy". But I get another problem. When I use logstash -f , can work to view "S" and "E" in kibana. When I use logstash -t -f and "Configuration OK", it not work.

filter{
date{
match=>["LogDateTime", "yyyy-MM-dd HH:mm:ss.SSSSSS"]
target=>"LogDateTime_2"
}
mutate{add_field=>{"ActionState"=>" "}}
if [Message]=="S"{
mutate{copy=>{"Message" => "ActionState"}}}
else if [Message]=="E"{
mutate{copy=>{"Message" => "ActionState"}}}
}

output{elasticsearch{hosts=>"localhost:9200"}}

[magnusbaeck (Magnus Bäck)]

When I use logstash -f , can work to view "S" and "E" in kibana. When I use logstash -t -f and "Configuration OK", it not work.

What do you mean? When you use -t Logstash will only parse the configuration and tell you if it looks okay. It won't process any data.

[zxc654951]

My recognition is wrong. Thanks.