Due to some underlying OS issues, seeing this messages that are quickly feeling up the master node logs, causing issues with disk availability
Caused by: RemoteTransportException[[prod566][10.3.10.86:9300][internal:gateway/local/started_shards[n]]]; nested: ElasticsearchException[failed to load started shards]; nested: NotSerializableExceptionWrapper[/home/t/elasticsearch-2.1.2/data/herd-dev-ev1/nodes/0/indices/f5-2016.06.13/0/_state: Too many open files in system];
What's the best way to handle this problem? Do i force the node out of the cluster? Usually, the data node is unresponsive, so I can't ssh to it.
My logging level at the moment is set to default: INFO
What I wanted to know more is to force that node to be taken out of rotation, but since I don't have ssh to the node at that moment to bring down elasticsearch, is there a way to dynamically issue this?
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.