An exception caught while applying mutate filter - Could not set field 'log' on object '' to value '...'

Elastic Stack Logstash
1

Hi,

I'm getting the following error message in Logstash logs:

[2020-08-11T15:07:11,248][WARN ][logstash.filters.mutate  ][main][51aae80b265c32978f64e0aace0ece634a29ae2a08ef228095111afd72bfdd0b] Exception caught while applying mutate filter {:exception=>"Could not set field 'log' on object '' to value '{\"offset\":7940658,\"file\":{\"path\":\"/var/log/containers/batch-processing-55555555-44sfk_default_batch-processing-a2053220bee1bda27497a272dce2555555555555557d0bc937d192f.log\"}}'.This is probably due to trying to set a field like [foo][bar] = someValuewhen [foo] is not either a map or a string"}

but I'm not quite sure why is it happening since that in the config I clearly check that the parent field doesn't exist before I add the subfield and set its value like this:

        filter {
    	  if ![metadata] {
      	    if [log][log] and ![log][log][offset] {
      	       mutate {
      		  add_field => { "[metadata][log]" => "%{[log][log]}" }
      	       }
      	    } else {
      	       mutate {
      		  add_field => { "[metadata][log]" => "%{log}" }
      	       }    		
      	    }
         }
      }

Any ideas?

2

Generally that exception is trying to tell you that your destination field is a concrete value, and you are trying to make it an object. The exception can be misleading because

  1. "on object ''" prints the value of the field, not the name of the field
  2. "is not either a map or a string" is just wrong, because the exception gets thrown when the field is a string!

So ... [metadata] exists, but is a value, not an object. Could it be a boolean?

3

Hi,

WOW, thanks for the quick reply. This is exactly where I'm puzzled... I wrapped all this code with if ![metadata] { to make sure that the field does not exist... or... am I missing something? Is it possible to tell Logstash to do the mutate stuff only when the field does not exist?

4

Suppose [metadata] were a boolean with the value false ... that test would evaluate true. I know it is the standard way to test whether a field exists, but the fact is there is no foolproof way of doing that in the logstash configuration syntax. As far as I know using a ruby filter is foolproof. Something like

ruby { code => 'event.set("metadataExists", event.to_hash.include?("metadata"))' }

will work.

5

so I'll add this and then I'll wrap the mutate code with a condition on whether metadataExists is false. right?

6

Yes.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.