Kibana not showing metrics/logs taken from ES. From /elasticsearch/logstash.log "reason transport disconnected (with verified connect)"


(Mariano González) #1

Hi guys,
We've implemented a log monitoring circle:
lumberjack-logstash-elasticsearch-logstash-kibana. Redis is correctly
recieving logs from logstash input, and sending logs to ES and logstash
filter. But there's an error in /elasticsearch/logstash.log:

[2013-11-28 20:58:23,918][INFO ][cluster.service ] [ES_server]
added
{[Infectia][mhAC9B-tTdunoKdCiTuIjg][inet[/logstash_filer_ip:9300]]{client=true,
data=false},}, reason: zen-disco-receive(join from
node[[Infectia][mhAC9B-tTdunoKdCiTuIjg][inet[/logstash_filter_ip:9300]]{client=true,
data=false}])
[2013-11-28 20:58:24,873][INFO ][cluster.service ] [ES_server]
removed
{[Infectia][mhAC9B-tTdunoKdCiTuIjg][inet[/logstash_filter_ip300]]{client=true,
data=false},}, reason:
zen-disco-node_failed([Infectia][mhAC9B-tTdunoKdCiTuIjg][inet[/logstas_filter_ip:9300]]{client=true,
data=false}), reason transport disconnected (with verified connect)

I've added a net.ipv4.tcp_keepalive_time = 1800 in sysctl.conf file (as
seen in a previous post) but it keeps on shooting the above error.

Is there a way to troubleshoot this? or has anybody got stuck with this?

Thanks!!

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/c5fbb7c1-f003-4af7-b322-9d9a0b2b1132%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Radu Gheorghe) #2

Hi Mariano,

Just shooting two ideas, I'm not sure if they'll help:

  1. Check that your version of ES matches the one Logstash is built on. For
    example, Logstash 1.2.2 needs ES 0.90.3
  2. If you can't get the "regular" elasticsearch plugin to work, try
    elasticsearch_http:
    http://logstash.net/docs/1.2.2/outputs/elasticsearch_http

Best regards,
Radu

On Thu, Nov 28, 2013 at 11:03 PM, Mariano González <
gonzalez.mariano.gabriel@gmail.com> wrote:

Hi guys,
We've implemented a log monitoring circle:
lumberjack-logstash-elasticsearch-logstash-kibana. Redis is correctly
recieving logs from logstash input, and sending logs to ES and logstash
filter. But there's an error in /elasticsearch/logstash.log:

[2013-11-28 20:58:23,918][INFO ][cluster.service ] [ES_server]
added
{[Infectia][mhAC9B-tTdunoKdCiTuIjg][inet[/logstash_filer_ip:9300]]{client=true,
data=false},}, reason: zen-disco-receive(join from
node[[Infectia][mhAC9B-tTdunoKdCiTuIjg][inet[/logstash_filter_ip:9300]]{client=true,
data=false}])
[2013-11-28 20:58:24,873][INFO ][cluster.service ] [ES_server]
removed
{[Infectia][mhAC9B-tTdunoKdCiTuIjg][inet[/logstash_filter_ip300]]{client=true,
data=false},}, reason:
zen-disco-node_failed([Infectia][mhAC9B-tTdunoKdCiTuIjg][inet[/logstas_filter_ip:9300]]{client=true,
data=false}), reason transport disconnected (with verified connect)

I've added a net.ipv4.tcp_keepalive_time = 1800 in sysctl.conf file (as
seen in a previous post) but it keeps on shooting the above error.

Is there a way to troubleshoot this? or has anybody got stuck with this?

Thanks!!

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/c5fbb7c1-f003-4af7-b322-9d9a0b2b1132%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
Performance Monitoring * Log Analytics * Search Analytics
Solr & Elasticsearch Support * http://sematext.com/

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAHXA0_1JS3uAYmG_H1g_tyo%2BXpmKbAg7%2B2r-QK7%3DTYHCdxegOw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Mariano González) #3

Thanks Radu.
We've resolved the issue by reinstalling a two node ES cluster. Seems like
this message was not triggering the major problem we had. Anyway it's
solved now. Thanks!!

On 2 December 2013 07:02, Radu Gheorghe radu.gheorghe@sematext.com wrote:

Hi Mariano,

Just shooting two ideas, I'm not sure if they'll help:

  1. Check that your version of ES matches the one Logstash is built on. For
    example, Logstash 1.2.2 needs ES 0.90.3
  2. If you can't get the "regular" elasticsearch plugin to work, try
    elasticsearch_http:
    http://logstash.net/docs/1.2.2/outputs/elasticsearch_http

Best regards,
Radu

On Thu, Nov 28, 2013 at 11:03 PM, Mariano González <
gonzalez.mariano.gabriel@gmail.com> wrote:

Hi guys,
We've implemented a log monitoring circle:
lumberjack-logstash-elasticsearch-logstash-kibana. Redis is correctly
recieving logs from logstash input, and sending logs to ES and logstash
filter. But there's an error in /elasticsearch/logstash.log:

[2013-11-28 20:58:23,918][INFO ][cluster.service ] [ES_server]
added
{[Infectia][mhAC9B-tTdunoKdCiTuIjg][inet[/logstash_filer_ip:9300]]{client=true,
data=false},}, reason: zen-disco-receive(join from
node[[Infectia][mhAC9B-tTdunoKdCiTuIjg][inet[/logstash_filter_ip:9300]]{client=true,
data=false}])
[2013-11-28 20:58:24,873][INFO ][cluster.service ] [ES_server]
removed
{[Infectia][mhAC9B-tTdunoKdCiTuIjg][inet[/logstash_filter_ip300]]{client=true,
data=false},}, reason:
zen-disco-node_failed([Infectia][mhAC9B-tTdunoKdCiTuIjg][inet[/logstas_filter_ip:9300]]{client=true,
data=false}), reason transport disconnected (with verified connect)

I've added a net.ipv4.tcp_keepalive_time = 1800 in sysctl.conf file (as
seen in a previous post) but it keeps on shooting the above error.

Is there a way to troubleshoot this? or has anybody got stuck with this?

Thanks!!

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/c5fbb7c1-f003-4af7-b322-9d9a0b2b1132%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
Performance Monitoring * Log Analytics * Search Analytics
Solr & Elasticsearch Support * http://sematext.com/

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/RPvWiQcYNIA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAHXA0_1JS3uAYmG_H1g_tyo%2BXpmKbAg7%2B2r-QK7%3DTYHCdxegOw%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
Mariano Gabriel González

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAHknVTVd2dOFXDvf7JPYzJ2XcEdJ4oP6cfHmrtcQWm7mQkYf%2BA%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(system) #4