I have a single node ES cluster with 26 GB RAM, 4 CPUs and 13 GB Heap Space. I am loading data into ES using threading in bulks of 5000 documents, the number of threads running can vary from 5 to 10. The problem that I am facing is that I am loading a
true/false flag which is being lost by ES. Every data field exists in the final documents except that particular flag. Some documents contain the flag and some don't. The ratio of exist to not exist is about 2:7. The piece of code that introduces that flag is a fairly simple if-else block
if foo: doc['flag'] = True else: doc['flag'] = False
I have created a mapping for the index and specifically set the field type as
boolean; I am unable to find a reason for the loss of the field. Any help would be greatly appreciated.
Note:- I am using ES 2.3.3