Logstash Not Receiving TCP Data

Elastic Stack Logstash
1

I am shipping monitored data as JSON from Python. I have tested whether the data is actually being sent outside of Logstash and it is successfully sending and being received. With Logstash the input is showing no signs of being received with the TCP input plugin.

Here is my configuration:

input{
   tcp{
     port => 55556
     codec => json
   }
}

output{
   elasticsearch {
   hosts => ['localhost:9200']
   sniffing => true
   index => "test2"
  document_type => "health"
  }
}

Verbose debugging doesn't show anything other than the basic output for logstash spinning up and connecting to the elasticsearch output specified. I'm under the impression that it might have something to do with the message being sent being ignored due to formatting.

Example message:

{"@fields": {"test": "test"}, "@message": {"doc_type": "sys_status", "PSUs": 2, "index": "shipper", "hostname": "client1", "CPUs": 2, "System": 4, "point_of_contact": "Tom Perry", "DIR": 4}, "@tags": ["test"]}
2

The input is definitely not valid JSON (which uses double quotes around strings, and does not support an encoding prefix, as strings are always UTF-8-encoded), but when the JSON codec fails to parse an event, the event is created anyway with the literal text of the message as its message attribute and _jsonparsefailure added to the event's tags.

Do you have evidence that TCP traffic is arriving?

tcpdump tcp port 55556
3

Sorry about that. Just realized that the example message was the message before a json.dumps() calls on it. I corrected it to what is actually sent.

Also, tcpdump does not show anything. I might be using it incorrectly, though.

4

Got tcpdump to show the proper packets with:

tcpdump -i any -n tcp dst port 55556

15:12:56.008867 IP 127.0.0.1.32886 > 127.0.0.1.55556: Flags [P.], seq 1020442282:1020442491, ack 201299404, win 342, options [nop,nop,TS val 21211295 ecr 21206290], length 209
15:13:01.014242 IP 127.0.0.1.32886 > 127.0.0.1.55556: Flags [P.], seq 209:672, ack 1, win 342, options [nop,nop,TS val 21216301 ecr 21211295], length 463
15:13:06.019543 IP 127.0.0.1.32886 > 127.0.0.1.55556: Flags [P.], seq 672:881, ack 1, win 342, options [nop,nop,TS val 21221306 ecr 21216301], length 209

Printing hex/ASCII shows they are the messages I am expecting to send.

5

Solved the issue. Json codec was automatically being converted to json_lines. After adding a new line after each message sent the index was created as well as messages being indexed in it.

6

Hi @clannadqs ,

Can you help me with this : tcp-data-sending-from-pyton-to-logstash-fails

Thanks

7

@clannadqs The same thing worked for me too :slight_smile:

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.