Want to try two different clusters sharing same data path (same index). The reason is, one cluster will keep on indexing streaming data. The other cluster will take care of all the searches and aggregations. So if at all the search cluster goes down due to memory hike on heavy aggregations, the data indexing will not get affected .
Tried the same in single server with two nodes. Each nodes has different "cluster.name" values.
Also set the "node.max_local_storage_nodes: 2" and the "path.data" pointing to same path in both the nodes.
But the problem is, the first started node creates a folder "0" inside the configured "path.data". The secondly started node creates folder "1". So obviously the data indexed by first node is not available to the second node. Tried running the second node with "node.data: false". But the problem still persists.
Is this approach possible ? Then what should be configuration so both the nodes will point to same index directory. Is there any other way to handle this scenario other than above mentioned one. ?
Any help/suggestions would heavily appreciated.
Thanks a lot.