I need help configuring an elastic search cluster. I would like to
have the index shared between to nodes via an NFS mount.
I have set the gateway.fs.location property to the NFS destination
and I see a directory is created for the cluster with indices and
metadata as subfolders. This looks okay.
However, it appears the directory specified in the path.data
property also contains an index. I didn't expect the path.data
directory to also contain the index. Why are there two copies of
Gateways are a way for ES to asynchronously snapshot cluster data to
a central location for easy restoration. But as long as you're using
filesystem-based index storage (as opposed to RAM-based), path.data
will always contain your index data too.
If you're starting more than one node in a particular path.data, ES
will create more node-level subdirectories so they don't conflict,
e.g., 0/, 1/, etc. I know this works with multiple nodes running on
the same machine, but I haven't tried it with different machines
accessing the same volume. In theory you should be able to stick
with local gateway and point path.data at the NFS mount and keep only
one copy of your data.
Unfortunately, this also comes with a cost of now not only having to
hit the disk, but do it over the network with other machines
contending for the same resource.