ES cluster becomes unresponsive

I'm having a really weird issue with a logging cluster still in its
infancy, and I've no idea how to tackle the problem.

Essentially, after a seemingly arbitray period of time, the cluster will
become unresponsive until the master node is restarted. I'm not indexing a
massive amount of logs at the moment, so I don't think the volume is the
problem, but mainly I can't figure out how to determine what the problem is.

I've looked in the logs in all nodes on the cluster (only 3 at present) and
done a query to see the cluster state, and it's fine:

{
"cluster_name" : "graylog2",
"status" : "green",
"timed_out" : false,
"number_of_nodes" : 4,
"number_of_data_nodes" : 3,
"active_primary_shards" : 6,
"active_shards" : 12,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 0
}

However my graylog2 web interface says it cannot connect to the cluster,
and using elasticsearch head I get no node information except that the
status is "green"

As I said, there's no error in the ES logs, but then I don't have any debug
logging turned on. How can I go about troubleshooting this? I don't think
the problem is related to graylog2, as I was previously trialing logstash +
kibana and the problem was the same there, which I thought was a limitation
of the software.

Any help appreciated

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Hey,

not sure about graylog, but doesnt it use the java transport client to
connect to elasticsearch? Maybe there is a version mismatch between your
elasticsearch node and the client.

--Alex

On Tue, May 28, 2013 at 10:19 AM, Frap lee@brig.gs wrote:

I'm having a really weird issue with a logging cluster still in its
infancy, and I've no idea how to tackle the problem.

Essentially, after a seemingly arbitray period of time, the cluster will
become unresponsive until the master node is restarted. I'm not indexing a
massive amount of logs at the moment, so I don't think the volume is the
problem, but mainly I can't figure out how to determine what the problem is.

I've looked in the logs in all nodes on the cluster (only 3 at present)
and done a query to see the cluster state, and it's fine:

{
"cluster_name" : "graylog2",
"status" : "green",
"timed_out" : false,
"number_of_nodes" : 4,
"number_of_data_nodes" : 3,
"active_primary_shards" : 6,
"active_shards" : 12,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 0
}

However my graylog2 web interface says it cannot connect to the cluster,
and using elasticsearch head I get no node information except that the
status is "green"

As I said, there's no error in the ES logs, but then I don't have any
debug logging turned on. How can I go about troubleshooting this? I don't
think the problem is related to graylog2, as I was previously trialing
logstash + kibana and the problem was the same there, which I thought was a
limitation of the software.

Any help appreciated

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.