Weird shard allocation behavior when applying settings via java client

ElasticSearch 1.5.2

My app dynamically applies settings to the ES cluster on start up. Before making these changes, it had been waiting on green status of the cluster before applying settings. We switched to waiting for yellow, were still seeing good results. However, in one of our environments we started seeing 1-2 primary shards not being allocated.

After switching the code back to look for green status, this allocation issue went away. We switched to yellow for performance reasons, it can taken up to 30s for all of the shards to be allocated. So I'm wondering, is this expected behavior?

If they are empty shards, then yes that is a little slow.
What do the master logs show?