Logstash - wrong codec in tcp input plugin?


#1

I'm using the newest version of Logstash 5.3.1.
While I am using a tcp input plugins, it shows that the real codec being used is not the same one with my config file.
To confirm this question, I add a log output in the source code of tcp input plugins - tcp.rb:

While my codec config is json, it actually uses json_lines:

While my codec config is plain, it actually uses line:

sorry for my poor english, hope you can understand my questions!


PLAIN Codec vs. #fix_streaming_codecs - Breaking Change
(Pier-Hugues Pellerin) #2

This because the #fix_streaming_codecs changes it for JSONLines and Lines, see https://github.com/elastic/logstash/blob/0d1011980efc2fdae9b8e8a261b8d04f6ff537de/logstash-core/lib/logstash/inputs/base.rb#L122-L135

Because in this input we assume that an event is separated by a new line, if you were using the plain or the json codec, the plugin would never know when or where to split the events and will just keep buffering data. For the user experience we make the assumptions that your events will be split by \n, but you can change that behavior in both codec by configuring the delimiter option, see the jsonlines documentation


PLAIN Codec vs. #fix_streaming_codecs - Breaking Change
#3

Thanks a lot! u solved my confusion


(system) #5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.