Kibana not showing metrics/logs taken from ES. From /elasticsearch/logstash.log "reason transport disconnected (with verified connect)"

Mariano_Gonzalez · November 28, 2013, 9:03pm

Hi guys,
We've implemented a log monitoring circle:
lumberjack-logstash-elasticsearch-logstash-kibana. Redis is correctly
recieving logs from logstash input, and sending logs to ES and logstash
filter. But there's an error in /elasticsearch/logstash.log:

[2013-11-28 20:58:23,918][INFO ][cluster.service ] [ES_server]
added
{[Infectia][mhAC9B-tTdunoKdCiTuIjg][inet[/logstash_filer_ip:9300]]{client=true,
data=false},}, reason: zen-disco-receive(join from
node[[Infectia][mhAC9B-tTdunoKdCiTuIjg][inet[/logstash_filter_ip:9300]]{client=true,
data=false}])
[2013-11-28 20:58:24,873][INFO ][cluster.service ] [ES_server]
removed
{[Infectia][mhAC9B-tTdunoKdCiTuIjg][inet[/logstash_filter_ip300]]{client=true,
data=false},}, reason:
zen-disco-node_failed([Infectia][mhAC9B-tTdunoKdCiTuIjg][inet[/logstas_filter_ip:9300]]{client=true,
data=false}), reason transport disconnected (with verified connect)

I've added a net.ipv4.tcp_keepalive_time = 1800 in sysctl.conf file (as
seen in a previous post) but it keeps on shooting the above error.

Is there a way to troubleshoot this? or has anybody got stuck with this?

Thanks!!

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/c5fbb7c1-f003-4af7-b322-9d9a0b2b1132%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

radu_gheorghe · December 2, 2013, 10:02am

Hi Mariano,

Just shooting two ideas, I'm not sure if they'll help:

Check that your version of ES matches the one Logstash is built on. For
example, Logstash 1.2.2 needs ES 0.90.3
If you can't get the "regular" elasticsearch plugin to work, try
elasticsearch_http:
Elasticsearch output plugin | Logstash Reference [8.11] | Elastic

Best regards,
Radu

On Thu, Nov 28, 2013 at 11:03 PM, Mariano González <
gonzalez.mariano.gabriel@gmail.com> wrote:

Hi guys,
We've implemented a log monitoring circle:
lumberjack-logstash-elasticsearch-logstash-kibana. Redis is correctly
recieving logs from logstash input, and sending logs to ES and logstash
filter. But there's an error in /elasticsearch/logstash.log:

[2013-11-28 20:58:23,918][INFO ][cluster.service ] [ES_server]
added
{[Infectia][mhAC9B-tTdunoKdCiTuIjg][inet[/logstash_filer_ip:9300]]{client=true,
data=false},}, reason: zen-disco-receive(join from
node[[Infectia][mhAC9B-tTdunoKdCiTuIjg][inet[/logstash_filter_ip:9300]]{client=true,
data=false}])
[2013-11-28 20:58:24,873][INFO ][cluster.service ] [ES_server]
removed
{[Infectia][mhAC9B-tTdunoKdCiTuIjg][inet[/logstash_filter_ip300]]{client=true,
data=false},}, reason:
zen-disco-node_failed([Infectia][mhAC9B-tTdunoKdCiTuIjg][inet[/logstas_filter_ip:9300]]{client=true,
data=false}), reason transport disconnected (with verified connect)

I've added a net.ipv4.tcp_keepalive_time = 1800 in sysctl.conf file (as
seen in a previous post) but it keeps on shooting the above error.

Is there a way to troubleshoot this? or has anybody got stuck with this?

Thanks!!

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/c5fbb7c1-f003-4af7-b322-9d9a0b2b1132%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
Performance Monitoring * Log Analytics * Search Analytics
Solr & Elasticsearch Support * http://sematext.com/

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAHXA0_1JS3uAYmG_H1g_tyo%2BXpmKbAg7%2B2r-QK7%3DTYHCdxegOw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Mariano_Gonzalez · December 2, 2013, 1:00pm

Thanks Radu.
We've resolved the issue by reinstalling a two node ES cluster. Seems like
this message was not triggering the major problem we had. Anyway it's
solved now. Thanks!!

On 2 December 2013 07:02, Radu Gheorghe radu.gheorghe@sematext.com wrote:

Hi Mariano,

Just shooting two ideas, I'm not sure if they'll help:

Check that your version of ES matches the one Logstash is built on. For
example, Logstash 1.2.2 needs ES 0.90.3

If you can't get the "regular" elasticsearch plugin to work, try
elasticsearch_http:
Elasticsearch output plugin | Logstash Reference [8.11] | Elastic

Best regards,
Radu

On Thu, Nov 28, 2013 at 11:03 PM, Mariano González <
gonzalez.mariano.gabriel@gmail.com> wrote:

Hi guys,
We've implemented a log monitoring circle:
lumberjack-logstash-elasticsearch-logstash-kibana. Redis is correctly
recieving logs from logstash input, and sending logs to ES and logstash
filter. But there's an error in /elasticsearch/logstash.log:

[2013-11-28 20:58:23,918][INFO ][cluster.service ] [ES_server]
added
{[Infectia][mhAC9B-tTdunoKdCiTuIjg][inet[/logstash_filer_ip:9300]]{client=true,
data=false},}, reason: zen-disco-receive(join from
node[[Infectia][mhAC9B-tTdunoKdCiTuIjg][inet[/logstash_filter_ip:9300]]{client=true,
data=false}])
[2013-11-28 20:58:24,873][INFO ][cluster.service ] [ES_server]
removed
{[Infectia][mhAC9B-tTdunoKdCiTuIjg][inet[/logstash_filter_ip300]]{client=true,
data=false},}, reason:
zen-disco-node_failed([Infectia][mhAC9B-tTdunoKdCiTuIjg][inet[/logstas_filter_ip:9300]]{client=true,
data=false}), reason transport disconnected (with verified connect)

I've added a net.ipv4.tcp_keepalive_time = 1800 in sysctl.conf file (as
seen in a previous post) but it keeps on shooting the above error.

Is there a way to troubleshoot this? or has anybody got stuck with this?

Thanks!!

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/c5fbb7c1-f003-4af7-b322-9d9a0b2b1132%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
Performance Monitoring * Log Analytics * Search Analytics
Solr & Elasticsearch Support * http://sematext.com/

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/RPvWiQcYNIA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAHXA0_1JS3uAYmG_H1g_tyo%2BXpmKbAg7%2B2r-QK7%3DTYHCdxegOw%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
Mariano Gabriel González

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAHknVTVd2dOFXDvf7JPYzJ2XcEdJ4oP6cfHmrtcQWm7mQkYf%2BA%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.