I know the general advice is to keep hardware as homogeneous as possible. However...
I have an existing 5 node ES cluster (v1.7.5) where each node has 32GB of RAM. ES gets 14GB of that for heap.
I would like to (for various reasons) add 2 additional nodes, with the only difference being that they have 64GB of ram. I would plan on making the heap size 30GB on the new nodes, and they would also be version 1.7.5). The cluster would then have 7 nodes total. 5 at 32, and 2 at 64.
I am not currently seeing memory issues, but the data is growing and I want to get out ahead of things. I would also like to soak test this new hardware configuration without committing to it fully. What kind of negatives could happen if I added 2 additional nodes that have double the RAM/heap? I'm fine letting ES balance the data across all nodes equally - I just want to make sure something terrible won't happen.
What do you think?