Need splitting incoming data

Hi all,

i try using split filter property to split incoming data from tcp port, this data has the line format of seven values plus time stamp separated by comma "," and terminated with "\n" as following :

[ v1,v2,v3,v4,v5,v6,v7,ts,\n ]

The rate of it is 100 line each 700 millisecond, the problem is that i just receive first line from each sending ( one line each 700 millisecond ) like this


the configure file is

input {
tcp {
host => ""
port => "12397"

stdin {}
filter {

csv {
separator => ","
columns => ["v1", "v2", "v3", "v4", "v5", "v6", "v7", "ts"]

mutate {convert => ["v1", "float"]}
mutate {convert => ["v2", "float"]}
mutate {convert => ["v3", "float"]}
mutate {convert => ["v4", "float"]}
mutate {convert => ["v5", "float"]}
mutate {convert => ["v6", "float"]}
mutate {convert => ["v7", "float"]}
mutate {convert => ["ts", "string"]}

output {
csv {
path => "/home/ahmed/Downloads/logstash-2.0.0/test2.csv"
fields => ["v1", "v2", "v3", "v4", "v5", "v6", "v7", "ts"]

stdout {}

Thank you very much for helping me


The default codec of the tcp input is line which emits one event per line received so what you want to accomplish should work out of the box without a split filter. I would do this do debug this further:

  1. Explicitly set codec => line just to be sure.
  2. Use a stdout { codec => rubydebug } output to get the raw event data.
  3. Test with synthetic data that I send to Logstash with e.g. netcat so I know exactly what Logstash should see.
  4. If necessary crank up logging with --verbose or --debug.
1 Like

Dear friend,

Kindly, i try with your suggestions, but the problem still exist, the output file just contains 1 line instead of 100 per each data sending, from your notice about tcp codec input, is it mean the codec just catch the first line and omit the other for each transmission ?

Thanks for your guide


As I said, the line codec is supposed to emit one event per line. I can't reproduce what you describe with Logstash 1.5.3. With this simple configuration,

input { tcp { port => 13383 } }
output { stdout { codec => rubydebug } }

I can send multi-line messages over a single TCP connection with

$ echo 'message one
> message two' | netcat -q 0 localhost 13383

and get the following from Logstash:

$ /opt/logstash/bin/logstash -f test.config 
Logstash startup completed
       "message" => "message one",
      "@version" => "1",
    "@timestamp" => "2015-11-25T12:21:31.011Z",
          "host" => "0:0:0:0:0:0:0:1"
       "message" => "message two",
      "@version" => "1",
    "@timestamp" => "2015-11-25T12:21:31.012Z",
          "host" => "0:0:0:0:0:0:0:1"
1 Like