Hello experts,
I have a new logstash cluster running with below versions:
- elasticsearch = "0.90.9"
- logstash = "1.3.3"
- Java version = "1.6.0_45"
- nodes -> 4 nodes [VM's] [log002(nxlog server, logstash server and elasticsearch node), 003,004,005 (elasticsearch nodes)].
My nxlog conf - http://pastebin.com/A0H9rwT8
my logstash conf - http://pastebin.com/UAj1k6K4
There are couple of problems:
- log002 seems registered as two nodes one is as log002 and other is as Glitch [The name changes everytime I restart logstash server]. There are continuous lines in elasticsearch logs from all the nodes [the below is from log003]:
Glitch log002.utils log003.utils log004.utils log005.utils --> as seen from elasticsearch head
[2014-02-12 22:57:14,930][WARN ][discovery.zen ] [log003.utils] received a join request for an existing node [[Glitch][6Vi1Np4ETXG1GRHIaR0z9A][inet[/10.10.0.3:9300]]{client=true, data=false}]
[2014-02-12 22:57:59,961][WARN ][discovery.zen ] [log003.utils] received a join request for an existing node [[Glitch][6Vi1Np4ETXG1GRHIaR0z9A][inet[/10.10.0.3:9300]]{client=true, data=false}]
[2014-02-12 22:58:44,993][WARN ][discovery.zen ] [log003.utils] received a join request for an existing node [[Glitch][6Vi1Np4ETXG1GRHIaR0z9A][inet[/10.10.0.3:9300]]{client=true, data=false}]
[2014-02-12 22:59:30,035][WARN ][discovery.zen ] [log003.utils] received a join request for an existing node [[Glitch][6Vi1Np4ETXG1GRHIaR0z9A][inet[/10.10.0.3:9300]]{client=true, data=false}]
- I'm getting a _jsonparsefailure in all the messages I see in kibana. Can you please suggest what is wrong ?
{"@source":"tcp://127.0.0.1:54427/","@tags":["_jsonparsefailure"],"@fields":{},"@timestamp":"2014-02-12T23:02:47.134Z","@source_host":"127.0.0.1","@source_path":"/","@message":"sourceFile\u0002\u0001\u0011\u0000\u0000\u0000/var/log/user.log\u0013\u0000SyslogFacilityValue\u0001\u0001\u0001\u0000\u000...