Problem with mutate - convert and mutate - rename

Elastic Stack Logstash
1

Hi all

I make a filter to intercept the log of squid.

I make a pattern with grok debug that run and convert the various fields in fields in elasticsearch.

I'm trying to apply ECS Schema and for 2 fields I have first apply mutate - convert and then apply mutate - rename.

I convert 2 fields from string to integer and then rename it.

I found that the 2 fields are correctly renamed but I have 2 strings.

the piece of mutate is the following

filter {
      mutate {
        convert => { "http_response_bytes" => "integer" }
        convert => { "process_elapsed" => "integer" }

        rename => { "http_response_bytes" => "http.response.bytes"} 
        rename => { "process_elapsed" => "process.elapsed"} 
  }
}

I'm trying in different mode. The only convert process_elapsed without rename it's work. If I change the position of rename and convert it's work but in my mind it's the same convert and then rename and rename and then convert.

What is the problem with my configuration? I spend different hours to debug and found the other solution.

Thank you
Franco

2

mutate does things in a fixed order, and rename comes before convert, so the fields no longer exist when it tries to convert them. Split it into two mutate filters.

3

Thank you @Badger. So If change my filter in this mode

filter {
      mutate {
        convert => { "http_response_bytes" => "integer" }
        convert => { "process_elapsed" => "integer" }
  }
      mutate {
        rename => { "http_response_bytes" => "http.response.bytes"} 
        rename => { "process_elapsed" => "process.elapsed"} 
  }
}

So the problem is the order, mutate doesn't follow the order with I use but the processing order.

In the log logstash-plain.log I don't found error. I expected to found a message like http_response_bytes doesn't exist or similar. Why?

Thank you
Franco

4

In most cases if the source field for a filter does not exist then the filter is a no-op, it does not log an error or tag on failure (for filters where that applies).

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.