I see that
memory is no longer an option for the
index.store.type . I'm not sure what the considerations were - but I have a specific problem that might benefit from putting the index into memory and I'm looking for advice.
We have an aggregation flow were we take raw data, aggregate and enrich it and then save it again. This can happen dozens or hundreds of times for a particular document. The problem is that normally this will thrash IO when the segments are merged. I was hoping to use memory indices so that I wouldn't have to think about the disk until the main processing is finished - then it would be copied to a regular disk-based index.
My usage pattern can tolerate data loss (I just re-run the aggregations).
Would a Ram disk help here? Are people using a ramdisk for ES for this or similar problems?
Are there changes in my configuration when storing on a disk vs ramdisk? Eg, would I still use doc_fields?
Any other helps on improving performance with many updates?