Elasticsearch: 1.7.4 ( We understand that it is pretty old version)
replicas: 2 (total 3 copies, including primary shard)
In our ES cluster setup with above configuration. We saw inconsistency in query response (either result with docs or none). Digging further we narrowed it down that our primary shard doesn't return result (i.e. when we use ?preference=_primary_first )
Looking closer at the docs on primary copy of shard and replicas. there is delta. Primary copy of shard has source of document enabled for all the fields while replicas doesn't have it. desired behavior is to not have source ON for all the fields.
Looking at mapping of that type with ?preference=_primary_first and without it. THere is no delta in mapping or setting of index
Upgrading ES version is not trivial in our application side. I would like to know if there was a bug around it which was noticed and fixed before I make stronger upgrade case.
I noticed this discussion: Inconsistent field mapping -- sometimes string, sometimes long which is not the case with us since mapping is registered same in all the copies of shard.