Great ideas David, Merci 
Well, that does not help much unfortunately.
I tried to reinit replicas anyway. No luck. Issue is that most of the searches are totally ok, but some searches are not and the number of returned results varies from one call to the other.
My only idea around is that, as I return more like 6M of results from 3(M, it gets truncated somewhere in the chain, probably at the shard level. I don't best results get good scores out of a shard and be lost when merging shards results.
Is that possible that a shard won't return all results for some reasons ? (memory issues, incomplete read of the shard because enough results were found, ...)
I'm desesperately trying to upgrade ES but reindexing is a pain, from 1.7.2 to 5.5.0, difficult to handle and migrate all the API calls.
First tries on migrated documents seems to get more consistent. I've also increased the number of shards.