Processing binary data files, not processing the end lines

Elastic Stack Logstash
1

I tried processing a binary data file (Format application/octet-stream; charset=binary).
Logstash is not processing the end lines, instead it starts processing from the almost middle of the file.
Please help. I have to process the latest lines.

2

What does "latest lines" mean for a binary file? In my world binary file don't have lines. They're just blobs of data. What kind of files are you trying to get Logstash to read? What's your configuration?

4

There is no such thing as a "binary" character set. Figuring out what character set actually is used could be helpful.

Your configuration looks generally sane (but see comments below) and since we seem to be dealing with text logs it's not obvious why Logstash would misbehave.

  • Instead if %{WORD:skip}, just say %{WORD}. Then you don't need to remove the skip field either.
  • Never use more than one GREEDYDATA or DATA pattern in a single expression. It can very easily backfire and result in weird matches.
5

wow.. that's some super advice. I will see why m getting wierd response.
Thanks for your reply.