# Beats not parsing \`message\` correctly with filebeat protocol
**URL:** https://discuss.elastic.co/t/beats-not-parsing-message-correctly-with-filebeat-protocol/368517
**Category:** Beats
**Tags:** filebeat
**Created:** [October 9, 2024, 11:06am UTC](https://discuss.elastic.co/t/beats-not-parsing-message-correctly-with-filebeat-protocol/368517 "2024-10-09T11:06:22Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@Adam\_Kral](https://discuss.elastic.co/u/Adam_Kral)
#### Post date: [October 9, 2024, 11:06am UTC](https://discuss.elastic.co/t/beats-not-parsing-message-correctly-with-filebeat-protocol/368517/1 "2024-10-09T11:06:22Z")
Hi, I've ran into an issue with filebeat parsing a json `message` field when using the `output.logstash`.
filebeat version: 8.15.2
logstash version: 8.13.0
The thing is, when I use:
```auto
processors:
- decode_json_fields:
fields: ["message"]
```
on a valid json log, the input in Logstash looks like this:
`"{test=field}"`
I would expect a valid json object:
`{"test": "field"}`
2 interesting points:
1. when you set the output of the filebeat to a local file instead of logstash, it outputs a valid JSON
2. when you use any other field than `message`, it also produces a valid JSON
I need to do the decode on filebeat side. Because in production, I am doing some other processing on the fields.
The current workaround is to use different field as target, e.g. `message-json`, but it is quite clumsy and it needs additional adjustments on the logstash side, which I want to avoid.
I have prepared repository with minimal setup [here](https://github.com/adamkral12/elastic-filebeat), you can test it there.
Please advice if & how can this be tackled.
Thank you.
---
### Author:  [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [October 9, 2024, 11:44am UTC](https://discuss.elastic.co/t/beats-not-parsing-message-correctly-with-filebeat-protocol/368517/2 "2024-10-09T11:44:46Z")
Hello, and welcome.
> [@Adam\_Kral](#):
>
> on a valid json log, the input in Logstash looks like this:
> `"{test=field}"`
Please share some example of this, it is not clear what is the issue here.
---
### Author:  [@Adam\_Kral](https://discuss.elastic.co/u/Adam_Kral)
#### Post date: [October 9, 2024, 1:04pm UTC](https://discuss.elastic.co/t/beats-not-parsing-message-correctly-with-filebeat-protocol/368517/3 "2024-10-09T13:04:28Z")
input file (log.log) for logstream:
```auto
{"test": "field"}
```
filebeat.yml config:
```auto
output.logstash:
hosts: ${LOGSTREAM_HOSTS}
filebeat.inputs:
- type: log
paths:
- /var/log/log.log
tags: ["test-tag"]
processors:
- decode_json_fields:
fields: ["message"]
```
logstash.conf:
```auto
input {
beats {
port => 443
ssl_enabled => false
}
}
output {
stdout {
codec => rubydebug {
metadata => true
}
}
}
```
expected output on logstash:
```auto
"message" => {
"test" => "field"
}
```
actual output on logstash:
```auto
"message" => "{test=field}"
```
check [repo](https://github.com/adamkral12/elastic-filebeat/) to reproduce
---
### Author:  [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [October 9, 2024, 1:49pm UTC](https://discuss.elastic.co/t/beats-not-parsing-message-correctly-with-filebeat-protocol/368517/4 "2024-10-09T13:49:47Z")
Can you please share the output you are getting? Copy the output in th screen or take a screenshot.
Unfortunately I cannot replicate it at the moment.
Also, you are using the `rubydebug` codec, so it will update the logs like this:
```auto
{
"message" => {
"test" => "field"
},
"@timestamp" => 2024-10-09T13:47:28.325083866Z,
"@version" => "1"
}
```
If you want it in a json format you need to use the `json` codec, which will output something like this:
```auto
{"message":{"teste":"field"},"@timestamp":"2024-10-09T13:48:22.032225940Z","@version":"1"}
```
---
### Author:  [@Adam\_Kral](https://discuss.elastic.co/u/Adam_Kral)
#### Post date: [October 9, 2024, 2:46pm UTC](https://discuss.elastic.co/t/beats-not-parsing-message-correctly-with-filebeat-protocol/368517/5 "2024-10-09T14:46:02Z")
after changing the codec to json, this is the output:
```auto
{"host":{"name":"c95347091bc4"},"@version":"1","@timestamp":"2024-10-09T14:42:16.596Z","event":{"original":"{test=field}"},"message":"{test=field}","log":{"offset":0,"file":{"path":"/var/log/log.log"}},"agent":{"id":"a0c54e93-72bd-478c-b39c-a8016e767ba2","type":"filebeat","version":"8.15.2","name":"c95347091bc4","ephemeral_id":"50f25619-11b7-4431-918c-30c9870ad3bf"},"ecs":{"version":"8.0.0"},"tags":["test-tag","beats_input_codec_plain_applied"],"input":{"type":"log"}}
```
aren't you able to replicate it via the repo I provided? (check the readme there)
---
### Author:  [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [October 9, 2024, 3:38pm UTC](https://discuss.elastic.co/t/beats-not-parsing-message-correctly-with-filebeat-protocol/368517/6 "2024-10-09T15:38:05Z")
> [@Adam\_Kral](#):
>
> aren't you able to replicate it via the repo I provided?
I didn't try, unfortunately I do not have time for it at the moment.
But the log you share has this information:
> "event":{"original":"{test=field}"},"message":"{test=field}"
This means that your original message is `{test=field}`, this is not a valid json.
I'm not sure what you are trying to achieve here, if you use the `decode_json_fields` filebeat will parse your json message, but if your message looks like this one, then it will not work because it is not a valid json.
Please share the content of the file: `/var/log/log.log`
I do not see any issue in Filebeat nor Logstash, they are doing what they are configured to do, but your original message is not a valid json.
---
### Author:  [@Adam\_Kral](https://discuss.elastic.co/u/Adam_Kral)
#### Post date: [October 9, 2024, 7:24pm UTC](https://discuss.elastic.co/t/beats-not-parsing-message-correctly-with-filebeat-protocol/368517/7 "2024-10-09T19:24:37Z")
That's why I believe there is an issue, the original log.log file looks like this:
```auto
{"test":"field"}
```
---
### Author:  [@Adam\_Kral](https://discuss.elastic.co/u/Adam_Kral)
#### Post date: [October 11, 2024, 11:54am UTC](https://discuss.elastic.co/t/beats-not-parsing-message-correctly-with-filebeat-protocol/368517/8 "2024-10-11T11:54:01Z")
Hello again, did you have any more time to look into this? 🙏
I've been debugging the beats go code and it is seems to be sending the body correctly
```auto
"message": {"test":"field"}
```
This would indicate to me, that logstash is treating the `message` field in a special way and somehow forcing the value to be:
```auto
"{test=field}"
```
Is it possible? If I send the data in any other field than `message`, it seems to be working as expected.
I am probably not able to dig into logstash deeper, since it's in Java and I would need too much time to set the tools up.