Beats TCP Message Body Decode

Elastic Stack Beats
1

I just have a Question I am working on a private service and want to integrate beat messages ingestion on my service, I been trying to decode tcp data which is coming from beat, I was successfully decoding the packet itself but unable to decode message body field.

I am using nodejs as a server side framework.

Now as I am new to TCP Am i missing something here?

2

What exactly do you mean by TCP Message Body? You try to have beats send to your own service (Which output are you using in beats)?

3

I am using logstash as output for my own service.

4

The message beats send to logstash might be compressed. You can find a go-lang based reference implementation of the lumberjack protocol (look for v2) at https://github.com/elastic/go-lumber

5

Thanks a lot for giving me the let me try and understand it first I will write after if I succeed or not.

6

I tried to use zlib but still can not decript, I Descripted Binary Packaet with binary decripters0 but still data field can not be decripted

7

Can you add some more details?

Compression in beats can be disabled (set compression: 0). First try to process data with compression disabled. Add compression support later.

1 Like
8

But I can not control compression option from other people if you have other suggestion ? as I will be using beats as a client for my server from third party clients

9

That's right, you can not enforce people to disable compression. The idea is to first get protocol support (sans compression) working. Just to remove the chance of some other protocol handling errors. Once you got the protocol working, adding compression is quite simple.

10

I just have one problem can not decode data send from lumberjack client

11

Without code and extensive debug logs on your server implementation I am not able to help in any way here. The events are encoded in JSON, embedded in the lumberjack framing. If you can not even decode anything I'd guess you have some error in the framing/offset handling.

12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.