graylog2 didn't recognize gelf output from logstash,
and still cann't find the way to fix it 。
but when i try nc test direct to graylog2 ,everything is ok。 just not work when get from logstash gelf output
when i trying tcpdump compare nc and logstash, the message show from the tcp stream totally not recognize but it's ok for nc test!
here is the some detail,really appreciate for any suggestion
a: my setup
logstash(gelf output)--------->--------graylog2(gelf udp input)
b: my log
55.3.244.1 GET /index.html 15824 0.043
c: logstash conf
input {
file {
path => "/etc/logstash/ces"
}
}
filter {
grok {
match => { "message" => "%{IP:client} %{WORD:method} %{URIPATHPARAM:request} %{NUMBER:bytes} %{NUMBER:duration}"}
}
}
output {
gelf {
host => "192.168.88.109"
codec => json
custom_fields => ["environment", "uat"]
}
stdout { codec => json }
}
d: logstash debug message (Version : 2.1.1)
https://groups.google.com/forum/?fromgroups=#!topic/graylog2/uq__WIOxI5s
tcpdump message when test different way
1: use nc test (ok)
echo -e '{"message":"55.3.244.1 GET /index.html 15824 0.043","@version":"1","@timestamp":"2014-08-17T19:36:45.825Z","host":"gz3.test","path":"/etc/logstash/ces","client":"55.3.244.1","method":"GET","request":"/index.html","bytes":"15824","duration":"0.043"}' | nc -u 192.168.88.109 12201
0.000000 192.168.88.64 192.168.88.109 UDP 291 Source port: 34983 Destination port: 12201
{"message":"55.3.244.1 GET /index.html 15824 0.043","@version":"1","@timestamp":"2014-08-17T19:36:45.825Z","host":"gz3.test","path":"/etc/logstash/ces","client":"55.3.244.1","method":"GET","request":"/index.html","bytes":"15824","duration":"0.043"}
2: use logstash test (not ok)
[root@gz3 logstash]# cat c
55.3.244.1 GET /index.html 15824 0.043
[root@gz3 logstash]# cat c >> ces
[root@gz3 logstash]#
32.343310 192.168.88.64 192.168.88.109 UDP 261 Source port: 34000 Destination port: 12201
x..P.N.0.......k....?.{.&.......(..gM%.Wn.3;3..`;..Rd#....T...]q..[.a.....-fr......C.......W6J-.%.~V|.t_l.x&...H.....\6./..sq.7n1.F..Q7....s.....|5hI..N.O+.Ul7A....D.cW..h.L0Y.=.m...~.T.r.BjO5....
..y...r.....d._..i.
3: use nc test (not ok)
echo -e '{"message"=>"55.3.244.1 GET /index.html 15824 0.043", "@version"=>"1", "@timestamp"=>"2014-08-17T22:11:13.806Z", "host"=>"gz3.test", "path"=>"/etc/logstash/ces", "client"=>"55.3.244.1", "method"=>"GET", "request"=>"/index.html", "bytes"=>"15824", "duration"=>"0.043"}' | nc -u 192.168.88.109 12201
208.101006 192.168.88.64 192.168.88.109 UDP 310 Source port: 41607 Destination port: 12201
{"message"=>"55.3.244.1 GET /index.html 15824 0.043", "@version"=>"1", "@timestamp"=>"2014-08-17T22:11:13.806Z", "host"=>"gz3.test", "path"=>"/etc/logstash/ces", "client"=>"55.3.244.1", "method"=>"GET", "request"=>"/index.html", "bytes"=>"15824", "duration"=>"0.043"}