# Beats TCP Message Body Decode

**URL:** <https://discuss.elastic.co/t/beats-tcp-message-body-decode/97674>\
**Category:** Beats\
**Tags:** beats-development\
**Created:** [August 20, 2017, 6:16am UTC](https://discuss.elastic.co/t/beats-tcp-message-body-decode/97674 "2017-08-20T06:16:45Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![Zeeshan\_Haider](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zeeshan_haider/32/64022_2.png) [@Zeeshan\_Haider](https://discuss.elastic.co/u/Zeeshan_Haider)\
**Post date:** [August 20, 2017, 6:16am UTC](https://discuss.elastic.co/t/beats-tcp-message-body-decode/97674/1 "2017-08-20T06:16:45Z")

</div>

I just have a Question I am working on a private service and want to integrate beat messages ingestion on my service, I been trying to decode tcp data which is coming from beat, I was successfully decoding the packet itself but unable to decode message body field.

I am using nodejs as a server side framework.

Now as I am new to TCP Am i missing something here?

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [August 20, 2017, 5:05pm UTC](https://discuss.elastic.co/t/beats-tcp-message-body-decode/97674/2 "2017-08-20T17:05:38Z")

</div>

What exactly do you mean by TCP Message Body? You try to have beats send to your own service (Which output are you using in beats)?

---

<div class="post-metadata">

**Author:** ![Zeeshan\_Haider](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zeeshan_haider/32/64022_2.png) [@Zeeshan\_Haider](https://discuss.elastic.co/u/Zeeshan_Haider)\
**Post date:** [August 21, 2017, 6:11am UTC](https://discuss.elastic.co/t/beats-tcp-message-body-decode/97674/3 "2017-08-21T06:11:09Z")

</div>

I am using logstash as output for my own service.

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [August 21, 2017, 2:13pm UTC](https://discuss.elastic.co/t/beats-tcp-message-body-decode/97674/4 "2017-08-21T14:13:17Z")

</div>

The message beats send to logstash might be compressed. You can find a go-lang based reference implementation of the lumberjack protocol (look for v2) at [https://github.com/elastic/go-lumber](https://github.com/elastic/go-lumber)

---

<div class="post-metadata">

**Author:** ![Zeeshan\_Haider](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zeeshan_haider/32/64022_2.png) [@Zeeshan\_Haider](https://discuss.elastic.co/u/Zeeshan_Haider)\
**Post date:** [August 22, 2017, 6:17am UTC](https://discuss.elastic.co/t/beats-tcp-message-body-decode/97674/5 "2017-08-22T06:17:03Z")

</div>

Thanks a lot for giving me the let me try and understand it first I will write after if I succeed or not.

---

<div class="post-metadata">

**Author:** ![Zeeshan\_Haider](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zeeshan_haider/32/64022_2.png) [@Zeeshan\_Haider](https://discuss.elastic.co/u/Zeeshan_Haider)\
**Post date:** [September 5, 2017, 8:40am UTC](https://discuss.elastic.co/t/beats-tcp-message-body-decode/97674/6 "2017-09-05T08:40:06Z")

</div>

I tried to use zlib but still can not decript, I Descripted Binary Packaet with binary decripters0 but still data field can not be decripted

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [September 5, 2017, 3:43pm UTC](https://discuss.elastic.co/t/beats-tcp-message-body-decode/97674/7 "2017-09-05T15:43:34Z")

</div>

Can you add some more details?

Compression in beats can be disabled (set `compression: 0`). First try to process data with compression disabled. Add compression support later.

---

<div class="post-metadata">

**Author:** ![Zeeshan\_Haider](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zeeshan_haider/32/64022_2.png) [@Zeeshan\_Haider](https://discuss.elastic.co/u/Zeeshan_Haider)\
**Post date:** [September 9, 2017, 6:39am UTC](https://discuss.elastic.co/t/beats-tcp-message-body-decode/97674/8 "2017-09-09T06:39:36Z")

</div>

But I can not control compression option from other people if you have other suggestion ? as I will be using beats as a client for my server from third party clients

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [September 11, 2017, 11:43am UTC](https://discuss.elastic.co/t/beats-tcp-message-body-decode/97674/9 "2017-09-11T11:43:38Z")

</div>

That's right, you can not enforce people to disable compression. The idea is to first get protocol support (sans compression) working. Just to remove the chance of some other protocol handling errors. Once you got the protocol working, adding compression is quite simple.

---

<div class="post-metadata">

**Author:** ![Zeeshan\_Haider](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zeeshan_haider/32/64022_2.png) [@Zeeshan\_Haider](https://discuss.elastic.co/u/Zeeshan_Haider)\
**Post date:** [September 19, 2017, 10:40am UTC](https://discuss.elastic.co/t/beats-tcp-message-body-decode/97674/10 "2017-09-19T10:40:31Z")

</div>

I just have one problem can not decode data send from lumberjack client

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [October 4, 2017, 10:49am UTC](https://discuss.elastic.co/t/beats-tcp-message-body-decode/97674/11 "2017-10-04T10:49:16Z")

</div>

Without code and extensive debug logs on your server implementation I am not able to help in any way here. The events are encoded in JSON, embedded in the lumberjack framing. If you can not even decode anything I'd guess you have some error in the framing/offset handling.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 1, 2017, 10:53am UTC](https://discuss.elastic.co/t/beats-tcp-message-body-decode/97674/12 "2017-11-01T10:53:41Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
