Unable to Send Stats to Monitoring Cluster

Elastic Stack Elasticsearch
1

I am having an issue with sending Marvel stats to our monitoring cluster,
which is in Microsoft Azure. We have a production cluster on premise, a
monitoring cluster in Azure, and a working VPN between them. I am able to
access the Marvel Dashboard of the monitoring cluster from both of the
production ES servers. The access lists allow the full IP stack between
each server. I am getting this error though on our prod servers:

[2014-04-22 07:05:03,359][ERROR][marvel.agent.exporter] [VERFY-MSPSVC02]
could not connect to any configured elasticsearch instances:
[VERIFY-AZ-SVC01:9200] [2014-04-22
07:05:03,370][ERROR][marvel.agent.exporter] [VERFY-MSPSVC02] error
connecting to [VERIFY-AZ-SVC01:9200] java.net.SocketTimeoutException:
connect timed out

This is what I have in my elasticsearch.yml:

marvel.agent.exporter.es.hosts: ["VERIFY-AZ-SVC01:9200"]

Any help would be greatly appreciated!!

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/aeb73320-45ba-482e-8d59-9b73b3eccade%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

Can you access the monitoring cluster from the command line using telnet
when logged in on the production machine? All marvel does is an HTTP POST
call..

Cheers,
Boaz

On Tuesday, April 22, 2014 5:13:29 PM UTC+2, Mario Rodriguez wrote:

I am having an issue with sending Marvel stats to our monitoring cluster,
which is in Microsoft Azure. We have a production cluster on premise, a
monitoring cluster in Azure, and a working VPN between them. I am able to
access the Marvel Dashboard of the monitoring cluster from both of the
production ES servers. The access lists allow the full IP stack between
each server. I am getting this error though on our prod servers:

[2014-04-22 07:05:03,359][ERROR][marvel.agent.exporter] [VERFY-MSPSVC02]
could not connect to any configured elasticsearch instances:
[VERIFY-AZ-SVC01:9200] [2014-04-22
07:05:03,370][ERROR][marvel.agent.exporter] [VERFY-MSPSVC02] error
connecting to [VERIFY-AZ-SVC01:9200] java.net.SocketTimeoutException:
connect timed out

This is what I have in my elasticsearch.yml:

marvel.agent.exporter.es.hosts: ["VERIFY-AZ-SVC01:9200"]

Any help would be greatly appreciated!!

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/9e468d30-5129-41fd-baea-66f6da667f96%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

3

Wednesday, April 23, 2014 7:54:50 AM UTC-5, Boaz Leskes wrote:

Can you access the monitoring cluster from the command line using telnet
when logged in on the production machine? All marvel does is an HTTP POST
call..

Cheers,
Boaz

Yes, I can connect to the instance using telnet.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/6cdc9e9c-bb51-4c61-8cd6-7b324d4540c6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

4

Just double checking - can you gist the marvel config part of your
elasticsearch.yml? the way it's pasted here will not be valid..

On Friday, April 25, 2014 10:16:31 PM UTC+2, Mario Rodriguez wrote:

Wednesday, April 23, 2014 7:54:50 AM UTC-5, Boaz Leskes wrote:

Can you access the monitoring cluster from the command line using telnet
when logged in on the production machine? All marvel does is an HTTP POST
call..

Cheers,
Boaz

Yes, I can connect to the instance using telnet.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/208229f5-72fd-49f1-b941-7f1382925507%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

5

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f278ddc2-edc6-421a-b2f0-abf699a36f4a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

6

https://gist.github.com/anonymous/11303362#file-gistfile1-txt - Prod Server
#1
https://gist.github.com/anonymous/11303451#file-gistfile1-txt - Monitoring
Server

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/9b2a7f82-036a-46a2-b738-e228dc7c1861%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

7

Hi Mario,

Gists look good to me.

One other thing I thought about - do you get this error all the time or
does appear every once in a while? Said differently, do you have data in
your monitoring cluster?

If so, you can try increasing the timeout (defaults to 6s):
marvel.agent.exporter.es.timeout: 30s

This can be done through the yml file or via the Cluster Update Settings
API :

Cheers,
Boaz

On Fri, Apr 25, 2014 at 11:15 PM, Mario Rodriguez starrio@gmail.com wrote:

VB ES Host #1 · GitHub - Prod
Server #1
Azure Monitoring ES Server · GitHub -
Monitoring Server

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/FNQf_hEXp04/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/9b2a7f82-036a-46a2-b738-e228dc7c1861%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/9b2a7f82-036a-46a2-b738-e228dc7c1861%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKzwz0qzZw3hzApLCSqqHOYyvMJpRo6eRS7dBx4-A3o5BYvaiQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

8

Hi Mario,

We just released marvel 1.1.1 with bug fix which I think will solve this

Can you check and see if it helps? (you can remove the 30s setting)

Cheers,
Boaz

On Sunday, April 27, 2014 10:12:31 PM UTC+2, Boaz Leskes wrote:

Hi Mario,

Gists look good to me.

One other thing I thought about - do you get this error all the time or
does appear every once in a while? Said differently, do you have data in
your monitoring cluster?

If so, you can try increasing the timeout (defaults to 6s):
marvel.agent.exporter.es.timeout: 30s

This can be done through the yml file or via the Cluster Update Settings
API :
Elasticsearch Platform — Find real-time answers at scale | Elastic

Cheers,
Boaz

On Fri, Apr 25, 2014 at 11:15 PM, Mario Rodriguez starrio@gmail.comwrote:

VB ES Host #1 · GitHub - Prod
Server #1
Azure Monitoring ES Server · GitHub -
Monitoring Server

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/FNQf_hEXp04/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/9b2a7f82-036a-46a2-b738-e228dc7c1861%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/9b2a7f82-036a-46a2-b738-e228dc7c1861%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d23bb576-4e78-4bac-9327-110c59905893%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.