# Filebeat 'dissect' won't override 'message' field

**URL:** https://discuss.elastic.co/t/filebeat-dissect-wont-override-message-field/374971
**Category:** Beats
**Tags:** filebeat
**Created:** [February 24, 2025, 10:37pm UTC](https://discuss.elastic.co/t/filebeat-dissect-wont-override-message-field/374971 "2025-02-24T22:37:30Z")
**Posts on this page:** 2
**Page:** 1

<div class="post-metadata">

### Author: ![yquirion](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yquirion/32/107068_2.png) [@yquirion](https://discuss.elastic.co/u/yquirion)
#### Post date: [February 24, 2025, 10:37pm UTC](https://discuss.elastic.co/t/filebeat-dissect-wont-override-message-field/374971/1 "2025-02-24T22:37:30Z")

</div>

Hi there!

I'm having a strange problem when using a `dissect` clause into `filebeat`.

Here is my code:

```auto
  - dissect:
      when:
        regexp:
          message: '^.*\s#remotelogmessage#\shost_ip:.*$'
      tokenizer: "%{[message]} #remotelogmessage# host_ip:%{[host][ip]}"
      field: "message"
      overwrite_keys: true
      target_prefix: ""

```

For an unknown reason, even if I specify `overwrite_keys: true`, the message is not being overwritten.

When using it with the above code, I'm having this in the `message` field:  
`[INFO][pulp.agent.b3421482-750d-4125-8226-d5e52ea060ca] gofer.messaging.adapter.connect:30 - connected: proton+amqps://satellite6.sti.usherbrooke.ca:5647 #remotelogmessage#`

I was able to confirm this by using another field name in the `tokenizer` settings. When replacing the `tokenizer` field by `grostata` instead of `message` like:

```auto
  - dissect:
      when:
        regexp:
          message: '^.*\s#remotelogmessage#\shost_ip:.*$'
      tokenizer: "%{[grostata]} #remotelogmessage# host_ip:%{[host][ip]}"
      field: "message"
      overwrite_keys: true
      target_prefix: ""

```

The field `grostata` will contain exactly what I want:  
`[INFO][pulp.agent.b3421482-750d-4125-8226-d5e52ea060ca] gofer.messaging.adapter.connect:30 - connected: proton+amqps://satellite6.sti.usherbrooke.ca:5647`

...which is removing a special tag I'm using for other purpose (`#remotelogmessage#`).

So, this proves that my `regex` is working fine. At first, I thought it was wrong, but I later noticed that the key wasn’t being overwritten.

As soon as I put back `message` instead of `grostata`, the tag `#remotelogmessage#` will be present into `message` field.

Is there anyone have an idea on this issue?

Thank you all and Best Regards,  
Yanick

---

<div class="post-metadata">

### Author: ![yquirion](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yquirion/32/107068_2.png) [@yquirion](https://discuss.elastic.co/u/yquirion)
#### Post date: [February 24, 2025, 11:15pm UTC](https://discuss.elastic.co/t/filebeat-dissect-wont-override-message-field/374971/2 "2025-02-24T23:15:13Z")

</div>

Dear all,

I found something interesting, but I don't have any explanation for the behaviour.

I was able to make the field `message` being overwritten. But to explain this correctly, I need to share more parts of my `filebeat.yml` configuration.

Here is a section for a specific filestream:

```auto
######################
# STI LOGS - MESSAGES
######################
- type: filestream

  # Change to true to enable this input configuration.
  enabled: true

  id: "sti-messages"

  # Paths that should be crawled and fetched. Glob based paths.
  paths:
    - /var/log/remote/*-messages.log

  tags: ["STI","system","remotelogmessage"]

  index: 'filebeat-sti-{{ beat_index_version }}-sys-linux'

  pipeline: filebeat-{{ beat_index_version }}-system-syslog-pipeline

  processors:
  - dissect:
      when:
        regexp:
          message: '^.*\s#remotelogmessage#\shost_ip:.*$'
      tokenizer: "%{[message]} #remotelogmessage# host_ip:%{[host][ip]}"
      field: "message"
      overwrite_keys: true
      target_prefix: ""

######################
# ANOTHER FILESTREAM SECTION STARTS BELOW
######################

```

The `processor` defined here shold only applied to this `filestream` section.

However, later in the configuration file, I have a "global" section, that should apply to all `filestream` I've defined above. Here is that section:

```auto
#####################
# Common parameters
#####################
tags: ["rsyslog"]

processors:
  - add_locale: ~

  - add_fields:
      target: ''
      fields:
        log_source: rsyslog
        client_name: sti  

  # Ce dissect sert à retirer le champs srcLogFile= qui a été ajouté par la configuration de rsyslog sur le client. 
  # Pour éviter des problèmes avec l'ingest pipeline, il faut retirer ce champs avant que la ligne arrive au logstash.
  - dissect:
      when:
        regexp:
          message: '^.*\ssrcLogFile=.*$'
      tokenizer: "%{} srcLogfile=%{srcLogfile}"  

  - dissect:
      when:
        regexp:
          message: '^.*\ssrcLogFile=.*$'
      tokenizer: "%{[message]} srcLogFile=%{[host][log][file][path]}"
      overwrite_keys: true
      target_prefix: ""

  - dissect:
      when:
        regexp:
          message: '^.*\shost_ip:.*$'
      tokenizer: "%{[message]} host_ip:%{[host][ip]}"
      field: "message"
      overwrite_keys: true
      target_prefix: ""

```

If I comment the last `dissect`:

```auto
 # - dissect:
 # when:
 # regexp:
 # message: '^.*\shost_ip:.*$'
 # tokenizer: "%{[message]} host_ip:%{[host][ip]}"
 # field: "message"
 # overwrite_keys: true
 # target_prefix: ""

```

then `message` field will be properly overwritten. The tag `#remotelogmessage#` will be removed.

Are those two `dissect` filters overwriting each other? Is there any specific order in which the `processors` are executed? Is the first `dissect` executing normally, but the change to the `message` field has not yet been committed, so the second `dissect` processes the original value?

The original input line looks like this:  
`Feb 24 18:04:18 kibana-prod01.domain.com kibana[1664090]: at TaskManagerRunner.run (/usr/share/kibana/node_modules/@kbn/task-manager-plugin/server/task_running/task_runner.js:309:22) #remotelogmessage# host_ip:10.132.204.99`

So the `#remotelogmessage#` has been stripped off, then the `host_ip` has been mapped to `[host][ip]` field and also stripped off the `message` field.

But if I remove the commented `dissect` section above, the tag `#remotelogmessage#` won't be stripped off the `message` field.

I understand it may be hard to figured out what's wrong here, but if you have an idea to share, please do so!

Regards,  
Yanick
