# Keeping \`message\` field intact with module parsing

**URL:** <https://discuss.elastic.co/t/keeping-message-field-intact-with-module-parsing/155452>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [November 5, 2018, 8:12pm UTC](https://discuss.elastic.co/t/keeping-message-field-intact-with-module-parsing/155452 "2018-11-05T20:12:23Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![rocketraman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rocketraman/32/32342_2.png) [@rocketraman](https://discuss.elastic.co/u/rocketraman)\
**Post date:** [November 5, 2018, 8:12pm UTC](https://discuss.elastic.co/t/keeping-message-field-intact-with-module-parsing/155452/1 "2018-11-05T20:12:23Z")

</div>

Parsing the message field with filebeat processors is great, as it allows structured search for logs, but one thing that surprises me is that the message field itself is destroyed. This means that just casual viewing of logs in Kibana or tools like [elktail](https://github.com/knes1/elktail) will just shown an empty log message. Here is an example from a Kibana dashboard:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/f/6/f626da6fd5dff3f442691048d1acaaee0bf86b58.png)

You then have to either add several of the destructured fields to the output, or click into the details of each one. This is not appealing when just trying to get an overall view of a set of logs before digging into the details.

In addition, if I search for `message:(something)` I won't find it. I have to know which destructured field contains `something` to do a search.

Can filebeat be configured to parse data out of the message, but leave the `message` field as-is rather than destroying it?

---

<div class="post-metadata">

**Author:** ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)\
**Post date:** [November 6, 2018, 10:22am UTC](https://discuss.elastic.co/t/keeping-message-field-intact-with-module-parsing/155452/2 "2018-11-06T10:22:41Z")

</div>

Which processor are you referring to? Processors `add_locale`, `add_*_metadata`, `include_fields` only enrich events with additional data, `drop_*` drops parts of the events or the whole event, targets of `decode_json_fields` and `dissect` are configurable.

---

<div class="post-metadata">

**Author:** ![rocketraman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rocketraman/32/32342_2.png) [@rocketraman](https://discuss.elastic.co/u/rocketraman)\
**Post date:** [November 6, 2018, 2:46pm UTC](https://discuss.elastic.co/t/keeping-message-field-intact-with-module-parsing/155452/3 "2018-11-06T14:46:50Z")

</div>

@kvch Sorry, I didn't specify: this is with filebeats running inside Kubernetes, with auto-discover enabled and hints on. The example logs I gave a screenshot above was with a pod annotated with:

```auto
co.elastic.logs/module: apache2
co.elastic.logs/fileset: access

```

---

<div class="post-metadata">

**Author:** ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)\
**Post date:** [November 6, 2018, 2:55pm UTC](https://discuss.elastic.co/t/keeping-message-field-intact-with-module-parsing/155452/4 "2018-11-06T14:55:38Z")

</div>

Sorry, I misunderstood. I thought you were referring to the processors of Filebeat, not the modules.  
Right now it is not supported. Do you mind opening an enhancement request? [https://github.com/elastic/beats/issues/new](https://github.com/elastic/beats/issues/new)

---

<div class="post-metadata">

**Author:** ![rocketraman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rocketraman/32/32342_2.png) [@rocketraman](https://discuss.elastic.co/u/rocketraman)\
**Post date:** [November 6, 2018, 5:10pm UTC](https://discuss.elastic.co/t/keeping-message-field-intact-with-module-parsing/155452/5 "2018-11-06T17:10:51Z")

</div>

> [@kvch](#):
>
> Do you mind opening an enhancement request?

Done: [Keep `message` field intact when using modules · Issue #8950 · elastic/beats · GitHub](https://github.com/elastic/beats/issues/8950)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 4, 2018, 5:10pm UTC](https://discuss.elastic.co/t/keeping-message-field-intact-with-module-parsing/155452/6 "2018-12-04T17:10:51Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.