# Share data directory between exclusively running instances

**URL:** <https://discuss.elastic.co/t/share-data-directory-between-exclusively-running-instances/219763>\
**Category:** Elasticsearch\
**Created:** [February 18, 2020, 10:43am UTC](https://discuss.elastic.co/t/share-data-directory-between-exclusively-running-instances/219763 "2020-02-18T10:43:12Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![noelzubinvictor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/noelzubinvictor/32/46090_2.png) [@noelzubinvictor](https://discuss.elastic.co/u/noelzubinvictor)\
**Post date:** [February 18, 2020, 10:43am UTC](https://discuss.elastic.co/t/share-data-directory-between-exclusively-running-instances/219763/1 "2020-02-18T10:43:12Z")

</div>

Problem: The ES instance require lot of memory for quick bulk updates. Otherwise its underutilized.

Can i create 2 ES instances with the same shared data directory. Say one with 4Gb(searching) and the other with 32Gb(indexing). and only run one of them at a time.

I could conditionally turn on the 64Gb one during indexing and replace it with the 4Gb one once its done

Would this work ?

---

<div class="post-metadata">

**Author:** ![Ayush\_Mathur](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ayush_mathur/32/77134_2.png) [@Ayush\_Mathur](https://discuss.elastic.co/u/Ayush_Mathur)\
**Post date:** [February 18, 2020, 1:12pm UTC](https://discuss.elastic.co/t/share-data-directory-between-exclusively-running-instances/219763/2 "2020-02-18T13:12:12Z")

</div>

The better option would be to configure hot-warm-cold architecture imho.

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [February 18, 2020, 4:18pm UTC](https://discuss.elastic.co/t/share-data-directory-between-exclusively-running-instances/219763/3 "2020-02-18T16:18:25Z")

</div>

What you describe is effectively running a single node with two different configs, and you are effectively asking whether you can change the config of a node. Yes, you can do that.

---

<div class="post-metadata">

**Author:** ![noelzubinvictor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/noelzubinvictor/32/46090_2.png) [@noelzubinvictor](https://discuss.elastic.co/u/noelzubinvictor)\
**Post date:** [February 18, 2020, 4:20pm UTC](https://discuss.elastic.co/t/share-data-directory-between-exclusively-running-instances/219763/4 "2020-02-18T16:20:48Z")

</div>

Well more like running two nodes with the same data directory isn't it ?

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [February 18, 2020, 4:24pm UTC](https://discuss.elastic.co/t/share-data-directory-between-exclusively-running-instances/219763/5 "2020-02-18T16:24:38Z")

</div>

Not really, no. If you only have one data directory then you only have one node, since a node is defined by the contents of its data directory. Everything else is config that is under your control.

---

<div class="post-metadata">

**Author:** ![noelzubinvictor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/noelzubinvictor/32/46090_2.png) [@noelzubinvictor](https://discuss.elastic.co/u/noelzubinvictor)\
**Post date:** [February 18, 2020, 4:26pm UTC](https://discuss.elastic.co/t/share-data-directory-between-exclusively-running-instances/219763/6 "2020-02-18T16:26:31Z")

</div>

Is it possible for 2 elasticsearch processes (running on different containers) to share the same data directory. Assuming only one them runs at a time.

---

<div class="post-metadata">

**Author:** ![noelzubinvictor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/noelzubinvictor/32/46090_2.png) [@noelzubinvictor](https://discuss.elastic.co/u/noelzubinvictor)\
**Post date:** [February 18, 2020, 4:31pm UTC](https://discuss.elastic.co/t/share-data-directory-between-exclusively-running-instances/219763/7 "2020-02-18T16:31:24Z")

</div>

Would having multiple nodes increase or decrease my indexing speed.

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [February 18, 2020, 7:49pm UTC](https://discuss.elastic.co/t/share-data-directory-between-exclusively-running-instances/219763/8 "2020-02-18T19:49:13Z")

</div>

> [@noelzubinvictor](#):
>
> Is it possible for 2 elasticsearch processes (running on different containers) to share the same data directory. Assuming only one them runs at a time.

Yes that's possible, again because you only have one data directory, hence one node, and you're just changing where it runs.

> [@noelzubinvictor](#):
>
> Would having multiple nodes increase or decrease my indexing speed.

It depends. Scaling out to multiple nodes is a good way to increase throughput, but there's overhead too.

---

<div class="post-metadata">

**Author:** ![noelzubinvictor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/noelzubinvictor/32/46090_2.png) [@noelzubinvictor](https://discuss.elastic.co/u/noelzubinvictor)\
**Post date:** [February 19, 2020, 5:34am UTC](https://discuss.elastic.co/t/share-data-directory-between-exclusively-running-instances/219763/9 "2020-02-19T05:34:49Z")

</div>

I will experiment again with other techniques and approaches to try increase my indexing speed. But if nothing else works. do you think the approach i mentioned above(shared data dir) would be a good solution for my quick indexing problem. Do you see any immediate issues that could happen with this approach.  
I haven't seen this way mentioned in any of the blogs or tutorials.

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [February 19, 2020, 8:14am UTC](https://discuss.elastic.co/t/share-data-directory-between-exclusively-running-instances/219763/10 "2020-02-19T08:14:14Z")

</div>

IMO the main drawback is just the downtime: it's not normally acceptable to restart things in between the indexing and searching phases, because people mostly want to search the data as it arrives. Furthermore if you want to do some more indexing then you must restart the node again, breaking any ongoing searches.

It's more usual to have a hot/warm setup, often with [ILM](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-lifecycle-management.html) to move indices from the hot tier to the warm tier and do various other optimisations like [force-merging](https://www.elastic.co/guide/en/elasticsearch/reference/current/_actions.html#ilm-forcemerge-action) them at the same time. That way you can scale the hot (indexing) and warm (search) tiers independently and can scale the hot tier down (all the way to zero if needed) when there's no indexing taking place, without affecting the warm tier.

---

<div class="post-metadata">

**Author:** ![noelzubinvictor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/noelzubinvictor/32/46090_2.png) [@noelzubinvictor](https://discuss.elastic.co/u/noelzubinvictor)\
**Post date:** [February 19, 2020, 8:46am UTC](https://discuss.elastic.co/t/share-data-directory-between-exclusively-running-instances/219763/11 "2020-02-19T08:46:13Z")

</div>

thanks man. Will check them out.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 18, 2020, 8:46am UTC](https://discuss.elastic.co/t/share-data-directory-between-exclusively-running-instances/219763/12 "2020-03-18T08:46:20Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
