# Filebeat kafka input message is incomplete and full of Unicode characters

**URL:** <https://discuss.elastic.co/t/filebeat-kafka-input-message-is-incomplete-and-full-of-unicode-characters/324139>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [January 27, 2023, 7:53pm UTC](https://discuss.elastic.co/t/filebeat-kafka-input-message-is-incomplete-and-full-of-unicode-characters/324139 "2023-01-27T19:53:17Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![gfar72](https://avatars.discourse-cdn.com/v4/letter/g/b38774/32.png) [@gfar72](https://discuss.elastic.co/u/gfar72)\
**Post date:** [January 27, 2023, 7:53pm UTC](https://discuss.elastic.co/t/filebeat-kafka-input-message-is-incomplete-and-full-of-unicode-characters/324139/1 "2023-01-27T19:53:17Z")

</div>

Hi Forum

We have a filebeat instance that uses the kafka input to read from a topic.  
The instance is able to connect to the topic, but the output of the "message" key is incomplete, and full of Unicode characters see the below example.

> "message": "\u0000\u0000\u0000\u0000\u0004Hfe4b2fd2-fd86-4b0e-a157-3ec3dc98f1b0\u0000\ufffd\u0001ebc0604a2673385a3b10b5a02ed4bbaf8c0135658efce5d7b3c4096a2c8f49f0Vsplit\_ICC\_head\_orientation\_90\_90\_mask\_train\u0004{}\u0002\nOwner\u001cMyuser\u0000\ufffd\u0001129f6167512d54e6db7f32f2cf38319e10c5868bc49780208d24158b32b40f72\ufffd\u0001FA30\_NIR\_\_2021-11-18\_15-21-51\_\_Session\_2.dmp\_00000283\_017d69b727\_0.tiffVsplit\_ICC\_head\_orientation\_90\_90\_mask\_train\u0006ICM${"pairs":"1 5053"}"

The source message have a "operation" key/value that is not shown in the message above so it is missing.

Any idea why the message content is full of the unicode characters?

Other kafka consumers show the message as a json string representation.

Thanks in advance

---

<div class="post-metadata">

**Author:** ![gfar72](https://avatars.discourse-cdn.com/v4/letter/g/b38774/32.png) [@gfar72](https://discuss.elastic.co/u/gfar72)\
**Post date:** [February 9, 2023, 11:19pm UTC](https://discuss.elastic.co/t/filebeat-kafka-input-message-is-incomplete-and-full-of-unicode-characters/324139/2 "2023-02-09T23:19:18Z")

</div>

Hi Here  
Just for future users reporting similar problem with filebeat and Kafka messages.  
If the kafka message topic is not serialized as string or JSON filebeat does not know how to rendered.  
So if you use custom serializers or AVRO you are out of luck with filebeat.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 10, 2023, 1:19am UTC](https://discuss.elastic.co/t/filebeat-kafka-input-message-is-incomplete-and-full-of-unicode-characters/324139/3 "2023-03-10T01:19:56Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.