Thank you very much for your answer.
I have been creating indices as daily log style, e.g. my-index-001-20160221
for handy management. And 001 is ranging from 001 ~ 100.
I guess this is a bad indexing strategy for memory usage.
Q. What if I disable norm for the fields? I am only filtering and aggregating for many of the fields. Would this help solving memory issue?
Q. What could be the side effect if I change my index pattern to be my-index-001-YYYYMM
? This will reduce the number of shards by 1/30
-ish but I would need to update/change daily docs when a day ends.