How to remove element value in array field

Elastic Stack Logstash
1

hi,
i want to remove an element from tags array in logstash .

logstash conf:

 filter {

     kv{

     source => "message"
     field_split => ","

     }

    date {
        match => [ "joiningdate", "MMMM dd yyyy HH:mm:ss.SSS", "yyyy-mm-dd", "yyyy.mm.dd" ]
  	}
    
    if "_dateparsefailure" in [tags] {
    	mutate {
    		add_field => { "JoiningDate" => "%{joiningdate}"}
    		remove_field => ["joiningdate"]
    	}
    }

    

 }

message : joiningdate=1234, name=asdf

output:

{
"host" => "xyz",
"JoiningDate" => "1234",
" name" => "asdf\r",
"message" => "joiningdate=1234, name=asdf\r",
"@version" => "1",
"@timestamp" => 2018-03-01T08:32:54.572Z,
"tags" => [
[0] "_dateparsefailure",
[1] "beats_input_code_plain_applied"
]
}

dateparsefailure can be in any index of tags array that should be removed.

tried tag_on_failure in date filter but in my logstash conf i am adding new field based on date parse failure. so pls suggest to remove _dateparsefailure from tags array dynamically

2

you can add a remove_tag => [ "_dateparsefailure" ] to your mutate filter.

2 Likes
3

Worked with
mutate {
remove_tag => [ "_dateparsefailure" ]
}

Thank you @pjanzen. But in my output i am getting empty tags field:

{
"@timestamp" => 2018-03-01T09:36:15.710Z,
" name" => "dhre\r",
"host" => "xyz",
"tags" => [],
"message" => "joiningdate=2q34, name=dhre\r",
"JoiningDate" => "2q34",
"@version" => "1"
}

here i want to remove empty array "tags". I tried

if [tags] == [] {
	mutate {
		remove_field => ["tags"]
	}
}

But got error :
[2018-03-01T15:07:56,000][FATAL][logstash.runner ] The given configuration is invalid. Reason: Failed to parse right-hand side of conditional [str]pipeline:31:8:```
[tags] == []

5

found solution.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.