# Message field - How become a valid field?

**URL:** <https://discuss.elastic.co/t/message-field-how-become-a-valid-field/220274>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [February 20, 2020, 10:20pm UTC](https://discuss.elastic.co/t/message-field-how-become-a-valid-field/220274 "2020-02-20T22:20:45Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Mauricio\_Borges](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mauricio_borges/32/54964_2.png) [@Mauricio\_Borges](https://discuss.elastic.co/u/Mauricio_Borges)\
**Post date:** [February 20, 2020, 10:20pm UTC](https://discuss.elastic.co/t/message-field-how-become-a-valid-field/220274/1 "2020-02-20T22:20:45Z")

</div>

Hi Team!  
We have a custom App and are trying define some Log Pattern based on some custom messages. At our Demo App where we type and submit any kind of test message, Filebeat outputs to ES and message appears at message field.  
However, when I try create some Visualization and use the "message" field over Terms, is not available.

What Can I do to have the message string show up as a valid field to create Visualizations on Kibana?

I have explored the following link, but not sure what condition or parameter to best achieve this goal. [https://www.elastic.co/guide/en/beats/filebeat/current/filtering-and-enhancing-data.html](https://www.elastic.co/guide/en/beats/filebeat/current/filtering-and-enhancing-data.html)

We are testing this at Openshift 4.x ( OCP on-preminse ) and deploying filebeat as daemonset.

Example of custom messages a Application can generate.  
"App01 - WebServer is starting"  
"App01 - WebServer is up and running"  
"App01 - WebServer is scaling 2 pods"  
"App02 - Database is will be restarted in 5 minutes"  
"App02 - Database is up and running"  
"App02 - Database is refreshing tables"

[Here](https://drive.google.com/file/d/1ZTtQo9jGMrjGo2hiw2ybfz65YZD75VZw/view?usp=sharing) more information about steps used.  
Here the [filebeat-kubernetes.yam](https://drive.google.com/file/d/1joZFnIaMwc6d2DD6qM8nQU8_3jyZWbMF/view?usp=sharing)l

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [February 26, 2020, 12:48am UTC](https://discuss.elastic.co/t/message-field-how-become-a-valid-field/220274/2 "2020-02-26T00:48:48Z")

</div>

The `message` field is an analyzed [`text` field](https://www.elastic.co/guide/en/elasticsearch/reference/master/text.html). If you want to do aggregations on the field then you should copy the data over to a different field that will be a [`keyword` data type](https://www.elastic.co/guide/en/elasticsearch/reference/master/keyword.html). `keyword` fields are supported in aggregations.

Based on those samples I'd recommend to use the dissect processor to split it up. It looks like the first part is a `service.name` and the second part could be a `service.status`. As an example:

```auto
processors:
- dissect:
    tokenizer: "%{service.name} - %{service.status}"
    field: "message"
    target_prefix: ""

```

---

<div class="post-metadata">

**Author:** ![dedemorton](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dedemorton/32/84409_2.png) [@dedemorton](https://discuss.elastic.co/u/dedemorton)\
**Post date:** [February 26, 2020, 2:52am UTC](https://discuss.elastic.co/t/message-field-how-become-a-valid-field/220274/3 "2020-02-26T02:52:24Z")

</div>

FYI...liked this example because it's simple and illustrates dissect really well. I've added it to the documentation:

> <https://github.com/elastic/beats/pull/16588>

---

<div class="post-metadata">

**Author:** ![Mauricio\_Borges](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mauricio_borges/32/54964_2.png) [@Mauricio\_Borges](https://discuss.elastic.co/u/Mauricio_Borges)\
**Post date:** [March 2, 2020, 11:40pm UTC](https://discuss.elastic.co/t/message-field-how-become-a-valid-field/220274/4 "2020-03-02T23:40:27Z")

</div>

Many thanks Andrew! It was very didactic!  
To start I have used the copy\_fields. Next step will check how App. Owner wish define it Pattern Log. Again very appreciated your help! Mauricio

processors:  
- add\_cloud\_metadata:  
- add\_host\_metadata:  
- add\_kubernetes\_metadata:  
- copy\_fields:  
fields:  
- from: message  
to: event.sms  
fail\_on\_error: false  
ignore\_missing: true  
when:  
contains:  
message: "SMS"

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 31, 2020, 1:40am UTC](https://discuss.elastic.co/t/message-field-how-become-a-valid-field/220274/5 "2020-03-31T01:40:28Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.