# Use alluxio as shared file system

**URL:** <https://discuss.elastic.co/t/use-alluxio-as-shared-file-system/178850>\
**Category:** Elasticsearch\
**Created:** [April 29, 2019, 7:13am UTC](https://discuss.elastic.co/t/use-alluxio-as-shared-file-system/178850 "2019-04-29T07:13:04Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![yiguolei](https://avatars.discourse-cdn.com/v4/letter/y/54ee81/32.png) [@yiguolei](https://discuss.elastic.co/u/yiguolei)\
**Post date:** [April 29, 2019, 7:13am UTC](https://discuss.elastic.co/t/use-alluxio-as-shared-file-system/178850/1 "2019-04-29T07:13:04Z")

</div>

Does elasticsearch will support alluxio as the underly shared filesystem. I think it has following advantages:

1. It will reduce cpu usage since data will be send only to primary replica and build index and the replicas will get the index segment from alluxio. And it is also the same during segment merge process.
2. Alluxio will cache data on local disk and it has short circuit read. It will not defect performance.
3. Elastic will become stateless and could be integrated with k8s more naturally.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [April 29, 2019, 7:16am UTC](https://discuss.elastic.co/t/use-alluxio-as-shared-file-system/178850/2 "2019-04-29T07:16:46Z")

</div>

> [@yiguolei](#):
>
> It will reduce cpu usage since data will be send only to primary replica and build index and the replicas will get the index segment from alluxio. And it is also the same during segment merge process.

Each node in Elasticsearch need to have its own copy of shards allocated to it so can not share data through a shared file system.

---

<div class="post-metadata">

**Author:** ![yiguolei](https://avatars.discourse-cdn.com/v4/letter/y/54ee81/32.png) [@yiguolei](https://discuss.elastic.co/u/yiguolei)\
**Post date:** [April 29, 2019, 7:29am UTC](https://discuss.elastic.co/t/use-alluxio-as-shared-file-system/178850/3 "2019-04-29T07:29:29Z")

</div>

In the past, Es supports shard file system. Primary replica could write data and follower replica implements a read only engine. They share same data on a shared file system.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [April 29, 2019, 7:51am UTC](https://discuss.elastic.co/t/use-alluxio-as-shared-file-system/178850/4 "2019-04-29T07:51:40Z")

</div>

If you are referring to shadow replicas, yes this feature has been removed. See discussion at

> <https://github.com/elastic/elasticsearch/issues/22024>

---

<div class="post-metadata">

**Author:** ![yiguolei](https://avatars.discourse-cdn.com/v4/letter/y/54ee81/32.png) [@yiguolei](https://discuss.elastic.co/u/yiguolei)\
**Post date:** [April 29, 2019, 8:13am UTC](https://discuss.elastic.co/t/use-alluxio-as-shared-file-system/178850/5 "2019-04-29T08:13:42Z")

</div>

Yes. It is shadow replica. In the past, shadow replica will read data from remote shared file system. Its performance maybe a little worse compared to local file system. But since alluxio could cache data on local file system. It will not defect performance. In the past, many deployment is in on promise environment and users could not find a very reliable shared filesystem. And it is also very difficult to manage both es and the shared file system.  
But there are many reliable storage system in public cloud and users do not need to manage them. I think it could reduce cost and simply es's architecture such as replication and recovery.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [April 29, 2019, 9:12am UTC](https://discuss.elastic.co/t/use-alluxio-as-shared-file-system/178850/6 "2019-04-29T09:12:47Z")

</div>

So the answer is that feature have been removed.  
I don't think it will come back again as it was (ie for live indices). But with hot/warm/cold architecture, you can probably store your cold indices on whatever shared FS may be and use the freeze API ([https://www.elastic.co/guide/en/elasticsearch/reference/7.0/freeze-index-api.html](https://www.elastic.co/guide/en/elasticsearch/reference/7.0/freeze-index-api.html))?

See also: [https://www.elastic.co/guide/en/elasticsearch/reference/7.0/frozen-indices.html](https://www.elastic.co/guide/en/elasticsearch/reference/7.0/frozen-indices.html)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 27, 2019, 9:12am UTC](https://discuss.elastic.co/t/use-alluxio-as-shared-file-system/178850/7 "2019-05-27T09:12:53Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
