Unable to make Cluster state to Green

Hello Team,
I am using two nodes in same host but am unable to see the cluster state as green,

Here is my response for cluster health
GET /_cluster/health?pretty=true

{
"cluster_name": "elasticsearch",
"status": "yellow",
"timed_out": false,
"number_of_nodes": 2,
"number_of_data_nodes": 2,
"active_primary_shards": 51,
"active_shards": 51,
"relocating_shards": 0,
"initializing_shards": 0,
"unassigned_shards": 51,
"number_of_pending_tasks": 0
}

Can anyone suggest me on this?????

You probably don't have enough disk space.
I think you can see it in logs...

Hello David,

Here are my logs..

[2016-01-19 17:09:36,614][INFO ][node                     ] [Franz Kafka1] initializing ...
[2016-01-19 17:09:36,784][INFO ][com.floragunn.searchguard.SearchGuardPlugin] Class enhancements for DLS/FLS successful
[2016-01-19 17:09:36,786][INFO ][plugins                  ] [Franz Kafka1] loaded [analysis-phonetic, searchguard (disabled)], sites [migration]
[2016-01-19 17:09:39,954][INFO ][node                     ] [Franz Kafka1] initialized
[2016-01-19 17:09:40,011][INFO ][node                     ] [Franz Kafka1] starting ...
[2016-01-19 17:09:40,071][INFO ][transport                ] [Franz Kafka1] bound_address {inet[/192.168.1.59:9300]}, publish_address {inet[/192.168.1.59:9300]}
[2016-01-19 17:09:40,079][INFO ][discovery                ] [Franz Kafka1] elasticsearch/SPI6MqvpREmv6ewlmO1rGQ
[2016-01-19 17:09:44,100][INFO ][cluster.service          ] [Franz Kafka1] new_master [Franz Kafka1][SPI6MqvpREmv6ewlmO1rGQ][Anusha-PC][inet[/192.168.1.59:9300]]{master=true}, reason: zen-disco-join (elected_as_master)
[2016-01-19 17:09:44,164][INFO ][http                     ] [Franz Kafka1] bound_address {inet[/192.168.1.59:9200]}, publish_address {inet[/192.168.1.59:9200]}
[2016-01-19 17:09:44,165][INFO ][node                     ] [Franz Kafka1] started
[2016-01-19 17:09:44,706][INFO ][cluster.routing.allocation.decider] [Franz Kafka1] updating [cluster.routing.allocation.disable_allocation] from [false] to [true]
[2016-01-19 17:09:44,707][INFO ][indices.ttl              ] [Franz Kafka1] updating indices.ttl.interval from [1m] to [15s]
[2016-01-19 17:09:44,814][INFO ][gateway                  ] [Franz Kafka1] recovered [11] indices into cluster_state
[2016-01-19 17:10:00,179][INFO ][cluster.service          ] [Franz Kafka1] added {[Franz Kafka2][UqFUB2KbQJa8r6ygz2s4zw][Gowthami-PC][inet[/192.168.1.45:9300]]{master=true},}, reason: zen-disco-receive(join from node[[Franz Kafka2][UqFUB2KbQJa8r6ygz2s4zw][Gowthami-PC][inet[/192.168.1.45:9300]]{master=true}])
[2016-01-19 17:12:27,863][INFO ][cluster.service          ] [Franz Kafka1] removed {[Franz Kafka2][UqFUB2KbQJa8r6ygz2s4zw][Gowthami-PC][inet[/192.168.1.45:9300]]{master=true},}, reason: zen-disco-node_left([Franz Kafka2][UqFUB2KbQJa8r6ygz2s4zw][Gowthami-PC][inet[/192.168.1.45:9300]]{master=true})
[2016-01-19 19:00:33,891][INFO ][bootstrap                ] running graceful exit on windows
[2016-01-19 19:00:33,893][INFO ][node                     ] [Franz Kafka1] stopping ...
[2016-01-19 19:00:33,997][INFO ][node                     ] [Franz Kafka1] stopped
[2016-01-19 19:00:33,997][INFO ][node                     ] [Franz Kafka1] closing ...
[2016-01-19 19:00:34,000][INFO ][node                     ] [Franz Kafka1] closed

Am unable to find disk space in logs....

Hello David, here are my indexes and the space used by them, I hope this data I useful to you,

health status index                                pri rep     docs.count    docs.deleted     store.size   pri.store.size 
yellow open   othermotive_parttype     5    1        435447            0                       73.9mb         73.9mb 
yellow open   dcl_raac_json                    5   1             9366             0                          1.8mb          1.8mb 
yellow open   master_index                   5   1                    0             0                             575b           575b 
yellow open   pdlfull                                5   1             5072             0                               1mb            1mb 
yellow open   dcl_raac_pcdb                  5   1            20241            0                             9.7mb          9.7mb 
yellow open   ymme                                5   1            35451            0                            25.3mb         25.3mb 
yellow open   demoymme                      5   1            35451            0                            16.8mb         16.8mb 
yellow open   ast                                      5   1                 336            0                            65.6kb         65.6kb 
yellow open   phonetic_sample             5   1                 314            0                            70.5kb         70.5kb 
yellow open   twitter                                1   1                 314            0                            31.7kb         31.7kb 
yellow open   othermotive_ymme        5   1             14396            0                              9.6mb          9.6mb

It shows that your second node has been removed 2 minutes after it appeared.

Why did you stop it?
Why did you stop the first node as well?

When did you run the cluster health? In the mean time?

No that was like my last trail to make a two node cluster, normally I just pasted the logs that am getting when two nodes are connected..
that's it , am getting only those logs apart from the start and stop logs..

I have started two nodes and verified the cluster state got as yellow.....
Do I need to do anything more than this???

For you to know that how much space occupied by the indexes I pasted the indexes and the sizes they occupied,
So is that becaz of the more disk space occupied or due to any other reason???

And Yes I run the cluster health in the mean while..

Well the only time I saw my replicas not being allocated was when I was below 20% of available disk space.

I would:

I have my cluster settings as shown,
{ "persistent": {
"cluster": {
"routing": {
"allocation": {
"disable_allocation": "true"
}
}
},
"discovery": {
"zen": {
"publish_timeout": "30s"
}
},
"indices": {
"ttl": {
"interval": "15000"
}
}
},
"transient": {}
}

is it the cause????

Hello David,
I stopped two nodes and started them again, this time am getting few indexes as yellow and few are green

And am getting exception as

[2016-01-20 16:03:29,676][WARN ][indices.cluster ] [Franz Kafka2] [[phonetic_sample][0]] marking and sending shard failed due to [failed to create index]
org.elasticsearch.indices.IndexCreationException: [phonetic_sample] failed to create index
at org.elasticsearch.indices.IndicesService.createIndex(IndicesService.java:330)
at org.elasticsearch.indices.cluster.IndicesClusterStateService.applyNewIndices(IndicesClusterStateService.java:311)
at org.elasticsearch.indices.cluster.IndicesClusterStateService.clusterChanged(IndicesClusterStateService.java:180)
at org.elasticsearch.cluster.service.InternalClusterService$UpdateTask.run(InternalClusterService.java:467)
at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:188)
at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:158)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.elasticsearch.ElasticsearchIllegalArgumentException: failed to find token filter type [phonetic] for [my_metaphone]
at org.elasticsearch.index.analysis.AnalysisModule.configure(AnalysisModule.java:249)
at org.elasticsearch.common.inject.AbstractModule.configure(AbstractModule.java:60)
at org.elasticsearch.common.inject.spi.Elements$RecordingBinder.install(Elements.java:204)

... 8 more

Caused by: org.elasticsearch.common.settings.NoClassSettingsException: Failed to load class setting [type] with value [phonetic]
at org.elasticsearch.common.settings.ImmutableSettings.loadClass(ImmutableSettings.java:476)
at org.elasticsearch.common.settings.ImmutableSettings.getAsClass(ImmutableSettings.java:464)
at org.elasticsearch.index.analysis.AnalysisModule.configure(AnalysisModule.java:239)
... 16 more
Caused by: java.lang.ClassNotFoundException: org.elasticsearch.index.analysis.phonetic.PhoneticTokenFilterFactory
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at org.elasticsearch.common.settings.ImmutableSettings.loadClass(ImmutableSettings.java:474)
... 18 more
[2016-01-20 16:03:29,686][WARN ][indices.cluster ] [Franz Kafka2] [[phonetic_sample][3]] marking and sending shard failed due to [failed to create index]
org.elasticsearch.indices.IndexCreationException: [phonetic_sample] failed to create index
at org.elasticsearch.indices.IndicesService.createIndex(IndicesService.java:330)
at org.elasticsearch.indices.cluster.IndicesClusterStateService.applyNewIndices(IndicesClusterStateService.java:311)
at org.elasticsearch.indices.cluster.IndicesClusterStateService.clusterChanged(IndicesClusterStateService.java:180)
at org.elasticsearch.cluster.service.InternalClusterService$UpdateTask.run(InternalClusterService.java:467)
at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:188)
at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:158)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
Caused by: org.elasticsearch.ElasticsearchIllegalArgumentException: failed to find token filter type [phonetic] for [my_metaphone]
at org.elasticsearch.index.analysis.AnalysisModule.configure(AnalysisModule.java:249)
... 8 more
Caused by: org.elasticsearch.common.settings.NoClassSettingsException: Failed to load class setting [type] with value [phonetic]
at org.elasticsearch.index.analysis.AnalysisModule.configure(AnalysisModule.java:239)
... 16 more
Caused by: java.lang.ClassNotFoundException: org.elasticsearch.index.analysis.phonetic.PhoneticTokenFilterFactory
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
... 18 more

Why suddenly the behaviour changed from unassigned to assign of few shards and few got exceptions?????

Probably you did not install the phonetic plugin on the second node?

Thanks David, Installed plugin works fine all went to Green... :relaxed: