Logstash/rsyslog/json

Elastic Stack Logstash
1

Hi All,

I was trying to setup rsyslog in elasticstack ,so I was trying to follow this article,


my logstash conf looks exactly similar to the article.

My logstash conf
,,,
input {
udp {
host => "private ip"
port => 5140
codec => "json"
workers => "16"
queue_size => "100000"
buffer_size => "100000"
type => "rsyslog"
}
}

This is an empty filter block. You can later add other filters here to further process

your log lines

filter { }

This output block will send all events of type "rsyslog" to Elasticsearch at the configured

host and port into daily indices of the pattern, "rsyslog-YYYY.MM.DD"

output {
if [type] == "rsyslog" {
stdout { codec => rubydebug }

elasticsearch {

hosts => [ "localhost:9200" ]

index => "logstash-%{+YYYY.MM.dd}"

}

}
}
,,,

after setting it up , I tried to run the logstash

bin/logstash -f /etc/logstash/conf.d/ --debug

I get this message

10:31:34.264 [Ruby-0-Thread-38: /usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:532] DEBUG logstash.pipeline - Pushing flush onto pipeline

I tried both stdout rubydebug output and elasticsearch output , still the same

I couldnt figure it out where the problem lies .Please help me to figure out this problem.

Thanks,
Raj

2

How do you know that any messages from remote servers are even reaching Logstash? You can use nc/netcat in listen mode to snoop the traffic (with Logstash shut down).

3

Thanks for the reply, I use tshark to confirm if i get the data

tshark -f 'port 5140'
tshark: Lua: Error during loading:
[string "/usr/share/wireshark/init.lua"]:46: dofile has been disabled due to running Wireshark as superuser. See http://wiki.wireshark.org/CaptureSetup/CapturePrivileges for help in running Wireshark as an unprivileged user.
Running as user "root" and group "root". This could be dangerous.
Capturing on 'eth0'
1 0.000000 -> TCP 78 57595→5140 [SYN] Seq=0 Win=32768 Len=0 MSS=1460 WS=8 SACK_PERM=1 TSval=1 TSecr=0
2 0.000023 -> TCP 54 5140→57595 [RST, ACK] Seq=1 Ack=1 Win=0 Len=0

4

So a TCP connection attempt (which is rejected) but you're only listening to UDP datagrams.

5

Yes Magnus, it seems to be only udp

6

Now Iam receiving docs in the Kibana but it looks this am not sure where the problem lies

image

I tried to tcpdump and check if there is data , most of data length is 0 and few are having data length of 3 and less values.

How to proceed with this?

Thanks,
Raj

7

Have you looked at what rsyslog actually is sending, to take Logstash out of the equation?

1 Like
8

there was some issues from client server side,now am able to receive it ,thanks for the help

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.