Currently everyday we write data to es with 8 datanodes, the wps is 50,000 , total 400 million records, during the writing process, there is hign burden for both cpu and io, and it affects the applications which read data from the es cluster. My question is Is there any way like HBase, we can leverage yarn to generate HFile, and load the file to HBase. Can we generate segments outside elasticsearch cluster, and load segments to cluster?
thanks