# Filter the content of "message" as independent fields

**URL:** https://discuss.elastic.co/t/filter-the-content-of-message-as-independent-fields/200260
**Category:** Beats
**Tags:** filebeat
**Created:** [September 19, 2019, 3:54pm UTC](https://discuss.elastic.co/t/filter-the-content-of-message-as-independent-fields/200260 "2019-09-19T15:54:34Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![David\_Oceans](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/david_oceans/32/51452_2.png) [@David\_Oceans](https://discuss.elastic.co/u/David_Oceans)
#### Post date: [September 19, 2019, 3:54pm UTC](https://discuss.elastic.co/t/filter-the-content-of-message-as-independent-fields/200260/1 "2019-09-19T15:54:34Z")

</div>

Hi!

I'm using filebeat in my GKE cluster. All my logs messages are in **json** format.

My surprise is that the **message** field is treated as a single field, and what I would like is for it to be treated as separate fields in order to have columns with the value key.

Example (extract) of json log received:

```
"_score": null,
  "_source": {
    "@timestamp": "2019-09-19T15:48:21.178Z",
    "message": "{\"target\":{\"requestID\":\"42170572-bea6-45fd-aba7-6cad3f75182b\",\"resourceID\":\"xxx-clique-v1-explorer-1\",\"actionPath\":\"/api/blocks?returnTransactions=true&returnReceipts=true&height=3080060\"},\"resourceInfo\":\"plugin=xxx-clique-v1-explorer-1\",\"authentication\":{\"identity\":{\"id\":12,\"identifier\":\"xxx-transactions-monitor-service-account@xxx-transactions-monitor-project.mserviceaccount.com\",\"maId\":\"61c577cc-77ab-4bd5-a369-e8b7bff583fa\",\"type\":\"SERVICE\",\"identityProfile\":{\"displayName\":\"xxxx TX Monitor Service Account\",\"avatarFilename\":\"\"},\"dateCreated\":1568886389000,\"dateUpdated\":1568886389000},\"project\":null,\"marpp\":{\"id\":8,\"name\":\"xxx Transactions Monitor\",\"marppId\":\"xxxx-transactions-monitor\",\"appToken\":\"\",\"organization\":\"abc-org\",\"imageURL\":\"https://storage.googleapis.com/xxx-img/image_marpp_8.png\",\"appURL\":\"https://txmonitor.xxxx.abc.com\"},\"valueTokenVersion\":1,\"iat\":1568908101,\"exp\":1568909001},\"topic\":\"plugin-proxy.request-arrival\"}",
    "log": {
      "offset": 8764173,
      "file": {
        "path": "/var/log/containers/resource-proxy-deployment-58886c655d-g8w4p_default_resource-proxy-6817ab68aaaa537ef0c25a65dbd747c7e8c9d7b9cbbb802f011dccc5751ed692.log"
      }
    },
    "input": {
      "type": "container"
    },

```

I would like filter by field " **topic**" that it is in message field.

That's its possible? I have to activate something in filebeat configuration?

Thank you so much

---

<div class="post-metadata">

### Author: ![kumarabhi](https://avatars.discourse-cdn.com/v4/letter/k/6a8cbe/32.png) [@kumarabhi](https://discuss.elastic.co/u/kumarabhi)
#### Post date: [September 23, 2019, 5:38pm UTC](https://discuss.elastic.co/t/filter-the-content-of-message-as-independent-fields/200260/2 "2019-09-23T17:38:06Z")

</div>

processors:

- decode\_json\_fields:  
fields: ["message"]  
target: ""  
overwrite\_keys: true

Reference : [Condition with decode\_json\_fields processor](https://discuss.elastic.co/t/condition-with-decode-json-fields-processor/115093/5)

---

<div class="post-metadata">

### Author: ![David\_Oceans](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/david_oceans/32/51452_2.png) [@David\_Oceans](https://discuss.elastic.co/u/David_Oceans)
#### Post date: [September 25, 2019, 8:10am UTC](https://discuss.elastic.co/t/filter-the-content-of-message-as-independent-fields/200260/3 "2019-09-25T08:10:08Z")

</div>

Thank you very much, works pretty well!

---

<div class="post-metadata">

### Author: ![kumarabhi](https://avatars.discourse-cdn.com/v4/letter/k/6a8cbe/32.png) [@kumarabhi](https://discuss.elastic.co/u/kumarabhi)
#### Post date: [September 25, 2019, 7:21pm UTC](https://discuss.elastic.co/t/filter-the-content-of-message-as-independent-fields/200260/4 "2019-09-25T19:21:06Z")

</div>

You may also want to convert fields to proper data types before sending to ES (you can check on ES side / Kibana the data type of fields). Ex : dateCreated can be converted to number and you can do range queries on that field.  
[https://www.elastic.co/guide/en/beats/filebeat/master/convert.html](https://www.elastic.co/guide/en/beats/filebeat/master/convert.html)

Cheers 🙂

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [October 23, 2019, 7:21pm UTC](https://discuss.elastic.co/t/filter-the-content-of-message-as-independent-fields/200260/5 "2019-10-23T19:21:06Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
