Restarting Elasticsearch when running along with logstash

Elastic Stack Elasticsearch
1

I started using Elastic Search very recently. So my questions might seen really idiotic. I have 2 es nodes and I am using logstash to send logs to them. The problem comes when I do a restart.
After I do a restart either both the ES nodes become masters and the cluster becoes red and they dont seem to identify the other. Error I got

 org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException: [logstash-2015.07.18][2] failed recovery
 at org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:162)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:745)
 Caused by: org.elasticsearch.index.engine.EngineCreationFailureException: [logstash-2015.07.18][2]     failed to upgrade 3x segments
 at org.elasticsearch.index.engine.InternalEngine.<init>(InternalEngine.java:121)
 at org.elasticsearch.index.engine.InternalEngineFactory.newReadWriteEngine(InternalEngineFactory.java:24)
 at org.elasticsearch.index.shard.IndexShard.newEngine(IndexShard.java:1251)
 at org.elasticsearch.index.shard.IndexShard.createNewEngine(IndexShard.java:1246)
 at org.elasticsearch.index.shard.IndexShard.prepareForTranslogRecovery(IndexShard.java:767)
 at org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:226)
 at org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:112)
2

That's not related to your dual master issue, to fix that you should really read https://www.elastic.co/guide/en/elasticsearch/guide/current/_important_configuration_changes.html

3

@warkolm Thanks for the quick response. I changed the settings and did a quick restart. I dont get that error now and also the cluster status is not red. On a completely different note I get this error in Logstash and it cannot connect to Elasticsearch sometimes.

Jul 18, 2015 8:14:33 AM org.elasticsearch.discovery.zen.ZenDiscovery joinElectedMaster
 failed to send join request to master  connect_timeout[30s]]; nested: SocketException[Network is unreachable]; ]
4

That means you have a problem with your networking.
Can you telnet from the LS host to the ES host on 9200?

5

They are both running in the same machine. In my logstash I have

elasticsearch { 
  host => localhost
  cluster => MyCluster
  ...
}
6

You need to put those in quotes, check the docs for formatting requirements :slight_smile:

7

Sorry They are already in quotes in the conf file, I did not copy them I just typed them for reference. Thanks for the info though.