Core vs extended field regarding a mutate error

[itschobot (Xavier)]

I'm getting a mutate error where the "host" value is being set to the "device_hostname"but was wondering if it's because one is a core field and the other is an extended field?

Here is what is being applied in my filter:

"host "=> "[host][hostname]"
"device_hostname" => "[host][id]"

That said, it seems to be the similar problem as this link correct? :

Error when copying a field to multiple others in a single mutate block

I noticed today, that copying the same field to multiple destinations inside the same mutate block, leads to a _mutate_error. Splitting the copy operation up to two mutate blocks works without any problem. Is this an intended behavior or should this be considered a bug? The following logstash.yml leads to a _mutate_error tag. input { generator { lines => [ 'May 28 11:11:01 threatarmor ta:BLOCKED_CONNECTION: UTC_TIME_MS="1590657056547" LOCAL_IP="1.1.1.1" …

Correct me if I'm wrong but rather than "copy" in the referenced article we are working with one "host" object due to "[host][hostname]" and "[host][id]" . Where one is a core field and the other is an extended, when my intent is to have them separate objects.

[Badger]

What is your mutate filter configuration?

[itschobot (Xavier)]

Here is my filter, it's the second mutate that I'm having issues with:

filter {
  if "some_platform" in [ls-source] {
   json { source => "message" }
   json { source => "message" }
   mutate { gsub => [ "message", "^[^{]+", "" ] }
   json { source => "message" }
  
  mutate{
    add_field => {
      "event_type" => "%{[fields][event_type]}"
      "log_offset" => "%{[log][offset]}"
      "log_filepath" => "%{[log][file][path]}"
      "agent_name" => "%{[agent][hostname]}"
      "agent_id" => "%{[agent][id]}"
      "agent_version" => "%{[agent][version]}"
      "ephemeral_id" => "%{[agent][ephemeral_id]}"
      "type" => "%{[agent][type]}"
      "device_label" => "%{[device][label]}"
      "device_ip" => "%{[device][ip]}"
      "device_hostname" => "%{[device][hostname]}"
      "device_macaddress" => "%{[device][macaddress]}"
      "device_vendor" => "%{[device][vendor]}"
      "priority" => "%{[model][priority]}"
      "model_description" => "%{[model][description]}"
      "ecs_version" => "%{[ecs][version]}"
   }
  }
  
  mutate{
   rename => {
    "@timestamp" => "[event][ingested]"
    "event_type" => "[event][type]"
    "ls-source" => "[event][category]"
    "priority" => "[event][risk_score]"
    "score" => "[event][risk_score_norm]"
    "host "=> "[host][hostname]"
    "device_hostname" => "[host][id]"
    "device_ip" => "[host][ip]"
    "device_macaddress" => "[device][mac]"
    "port" => "[source][port]"
    "device_vendor" => "[observer][vendor]"
    "device_label" => "[observer][product]"
    "agent_name" => "[agent][name]"
    "agent_id" => "[agent][id]"
    "agent_version" => "[agent][version]"
    "breachUrl" => "[threat][enrichments][indicator][url][full]"
    "model_description" => "[threat][enrichments][indicator][description]"
    "creationTime" => "[event][created]"
    "origin" => "[event][id]"
    "log_type" => "[event][kind]"
    "log_offset" => "[log][offset]"
    "ephemeral_id" => "[ephemeral][id]"
    "pbid" => "[service][id]"
    "log_filepath" => "[log][file][path]"
    "@version" => "[observer][version]"
    "ecs_version" => "[ecs][version]"
   }
  }
  
 }
}

[Badger]

Can you enable debug logging and tell us what gets logged here.

Also, it is unclear why you would

mutate { add_field => { "ecs_version" => "%{[ecs][version]}" } }
mutate { rename => { "ecs_version" => "[ecs][version]" } }

which is going to be a no-op if the original [ecs][version] field exists and is a string. If it is not a string it will get converted to a string. If it does not exist then it will get set to the literal string "%{[ecs][version]}".

The same applies to several other fields.

What is your ecs_compatibility setting? In particular, is [host] a string or an object?