# What are the requirements?

**URL:** <https://discuss.elastic.co/t/what-are-the-requirements/368279>\
**Category:** Elasticsearch\
**Created:** [October 5, 2024, 1:39am UTC](https://discuss.elastic.co/t/what-are-the-requirements/368279 "2024-10-05T01:39:39Z")\
**Posts on this page:** 20\
**Page:** 2

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [October 5, 2024, 5:58pm UTC](https://discuss.elastic.co/t/what-are-the-requirements/368279/21 "2024-10-05T17:58:14Z")

</div>

> [@dsagent](#):
>
> I understand from your words that I cannot transfer data from one storage space to another?

No, not within a single node. That is where the tiered architecture I described comes in. Did you read the blog post I linked to? That one is a bit old but you can easily find others through Google if you want a different take on this common architecture.

> [@dsagent](#):
>
> Well, if you make 14 nodes with SSD storage and another 14 with HDD storage  
> Then make the hot layer on the SDD and the cold layer on the HDD

That is the purpose f the architecture, but the node counts you specified sounds very low given the estimated data volumes. You can technically store a lot of data on a warm node with HDD, but be aware that the more data you store the slower queries will likely be.

---

<div class="post-metadata">

**Author:** ![dsagent](https://avatars.discourse-cdn.com/v4/letter/d/d2c977/32.png) [@dsagent](https://discuss.elastic.co/u/dsagent)\
**Post date:** [October 5, 2024, 6:11pm UTC](https://discuss.elastic.co/t/what-are-the-requirements/368279/22 "2024-10-05T18:11:35Z")

</div>

> [@Christian\_Dahlqvist](#):
>
> No, not within a single node. That is where the tiered architecture I described comes in. Did you read the blog post I linked to? That one is a bit old but you can easily find others through Google if you want a different take on this common architecture.

Yes, I read it.  
But what I meant is that even if I make the mentioned layers, I cannot transfer data from the SSD disk to the HDD after 14 days because I did not find anything to enter a value for the path of the other disk

---

<div class="post-metadata">

**Author:** ![dsagent](https://avatars.discourse-cdn.com/v4/letter/d/d2c977/32.png) [@dsagent](https://discuss.elastic.co/u/dsagent)\
**Post date:** [October 5, 2024, 6:16pm UTC](https://discuss.elastic.co/t/what-are-the-requirements/368279/23 "2024-10-05T18:16:12Z")

</div>

> [@Christian\_Dahlqvist](#):
>
> That is the purpose f the architecture, but the node counts you specified sounds very low given the estimated data volumes. You can technically store a lot of data on a warm node with HDD, but be aware that the more data you store the slower queries will likely be.

What is the number of nodes suitable for this operation?  
And what is the best scenario in your opinion?  
Because if something happens mistakes or something like that it will be bad

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [October 5, 2024, 6:20pm UTC](https://discuss.elastic.co/t/what-are-the-requirements/368279/24 "2024-10-05T18:20:13Z")

</div>

> [@dsagent](#):
>
> Yes, I read it.  
> But what I meant is that even if I make the mentioned layers, I cannot transfer data from the SSD disk to the HDD after 14 days because I did not find anything to enter a value for the path of the other disk

Nodes belonging to different tiers will be labelled and ILM (see other link I provided) is the process that will move indices between tiers based on configured policies.

> [@dsagent](#):
>
> What is the number of nodes suitable for this operation?  
> And what is the best scenario in your opinion?  
> Because if something happens mistakes or something like that it will be bad

I the data volume is accurate and you want to keep it for a full year I would recommend reaching out to Elastic for help with sizing as searchable snapshots will save you a lot of money on hardware and they can better help you size this use case.

---

<div class="post-metadata">

**Author:** ![dsagent](https://avatars.discourse-cdn.com/v4/letter/d/d2c977/32.png) [@dsagent](https://discuss.elastic.co/u/dsagent)\
**Post date:** [October 5, 2024, 6:24pm UTC](https://discuss.elastic.co/t/what-are-the-requirements/368279/25 "2024-10-05T18:24:23Z")

</div>

> [@Christian\_Dahlqvist](#):
>
> Nodes belonging to different tiers will be labelled and ILM (see other link I provided) is the process that will move indices between tiers based on configured policies.

Ok, I got it.

---

<div class="post-metadata">

**Author:** ![dsagent](https://avatars.discourse-cdn.com/v4/letter/d/d2c977/32.png) [@dsagent](https://discuss.elastic.co/u/dsagent)\
**Post date:** [October 5, 2024, 6:27pm UTC](https://discuss.elastic.co/t/what-are-the-requirements/368279/26 "2024-10-05T18:27:37Z")

</div>

> [@Christian\_Dahlqvist](#):
>
> I the data volume is accurate and you want to keep it for a full year I would recommend reaching out to Elastic for help with sizing as searchable snapshots will save you a lot of money on hardware and they can better help you size this use case.

Ok  
Thank you very much  
I will do that, but I don't think it's possible because the data is local.

---

<div class="post-metadata">

**Author:** ![dsagent](https://avatars.discourse-cdn.com/v4/letter/d/d2c977/32.png) [@dsagent](https://discuss.elastic.co/u/dsagent)\
**Post date:** [October 5, 2024, 6:29pm UTC](https://discuss.elastic.co/t/what-are-the-requirements/368279/27 "2024-10-05T18:29:20Z")

</div>

I have another question  
What about backing up if I do a snapshot can I take it and restore it in another set?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [October 5, 2024, 6:37pm UTC](https://discuss.elastic.co/t/what-are-the-requirements/368279/28 "2024-10-05T18:37:48Z")

</div>

You should be able to use searchable snapshots on premise as well as it is possible to shote snapshots on shared filesystem.

> [@dsagent](#):
>
> I have another question  
> What about backing up if I do a snapshot can I take it and restore it in another set?

If you can not fit the full data set in the cluster but still need access to parts of it you can store old data in a snapshot repository and restore old indices when needed. This will naturally take time and requires you to have enough spare capacity for the restore.

---

<div class="post-metadata">

**Author:** ![dsagent](https://avatars.discourse-cdn.com/v4/letter/d/d2c977/32.png) [@dsagent](https://discuss.elastic.co/u/dsagent)\
**Post date:** [October 5, 2024, 6:41pm UTC](https://discuss.elastic.co/t/what-are-the-requirements/368279/29 "2024-10-05T18:41:05Z")

</div>

> [@Christian\_Dahlqvist](#):
>
> If you can not fit the full data set in the cluster but still need access to parts of it you can store old data in a snapshot repository and restore old indices when needed. This will naturally take time and requires you to have enough spare capacity for the restore.

Yes, that's for sure. What I mean is that I made a Snapshot, for example, but there was a problem and I want to restore this shot from a group other than the group that I made the shot from

---

<div class="post-metadata">

**Author:** ![dsagent](https://avatars.discourse-cdn.com/v4/letter/d/d2c977/32.png) [@dsagent](https://discuss.elastic.co/u/dsagent)\
**Post date:** [October 5, 2024, 6:42pm UTC](https://discuss.elastic.co/t/what-are-the-requirements/368279/30 "2024-10-05T18:42:11Z")

</div>

> [@Christian\_Dahlqvist](#):
>
> shared filesystem

Yes, that's what I mean.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [October 5, 2024, 7:05pm UTC](https://discuss.elastic.co/t/what-are-the-requirements/368279/31 "2024-10-05T19:05:56Z")

</div>

I am not sure I understand. [Elasticsearch snapshots](https://www.elastic.co/guide/en/elasticsearch/guide/current/backing-up-your-cluster.html) are taken from the cluster as a whole and you can change settings when you restore.

---

<div class="post-metadata">

**Author:** ![dsagent](https://avatars.discourse-cdn.com/v4/letter/d/d2c977/32.png) [@dsagent](https://discuss.elastic.co/u/dsagent)\
**Post date:** [October 5, 2024, 7:30pm UTC](https://discuss.elastic.co/t/what-are-the-requirements/368279/32 "2024-10-05T19:30:43Z")

</div>

On what basis is the number of nodes determined?

---

<div class="post-metadata">

**Author:** ![dsagent](https://avatars.discourse-cdn.com/v4/letter/d/d2c977/32.png) [@dsagent](https://discuss.elastic.co/u/dsagent)\
**Post date:** [October 5, 2024, 7:35pm UTC](https://discuss.elastic.co/t/what-are-the-requirements/368279/33 "2024-10-05T19:35:23Z")

</div>

> [@Christian\_Dahlqvist](#):
>
> I am not sure I understand. [Elasticsearch snapshots](https://www.elastic.co/guide/en/elasticsearch/guide/current/backing-up-your-cluster.html) are taken from the cluster as a whole and you can change settings when you restore.

Of course I understand you.

I will explain more, for example, if I had a group and there was data in it and I made a snapshot, then there was a problem in the current group and I want to reproduce the shot except another group and restore it in the new group or to say I wanted to do some analysis and I wanted to restore the shot in a different group than the main group, can it be restored in the new group?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [October 5, 2024, 7:52pm UTC](https://discuss.elastic.co/t/what-are-the-requirements/368279/34 "2024-10-05T19:52:32Z")

</div>

I do not understand what you mean by group.

---

<div class="post-metadata">

**Author:** ![dsagent](https://avatars.discourse-cdn.com/v4/letter/d/d2c977/32.png) [@dsagent](https://discuss.elastic.co/u/dsagent)\
**Post date:** [October 5, 2024, 8:05pm UTC](https://discuss.elastic.co/t/what-are-the-requirements/368279/35 "2024-10-05T20:05:39Z")

</div>

> [@Christian\_Dahlqvist](#):
>
> I do not understand what you mean by group.

Cluster

---

<div class="post-metadata">

**Author:** ![dsagent](https://avatars.discourse-cdn.com/v4/letter/d/d2c977/32.png) [@dsagent](https://discuss.elastic.co/u/dsagent)\
**Post date:** [October 5, 2024, 8:07pm UTC](https://discuss.elastic.co/t/what-are-the-requirements/368279/36 "2024-10-05T20:07:18Z")

</div>

> [@dsagent](#):
>
> On what basis is the number of nodes determined?

Can you explain to me?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [October 5, 2024, 8:51pm UTC](https://discuss.elastic.co/t/what-are-the-requirements/368279/37 "2024-10-05T20:51:11Z")

</div>

All these tiers would be subsets of nodes within a single cluster. The snapshots are taken cluster-wide.

> [@dsagent](#):
>
> On what basis is the number of nodes determined?

For logging and metrics use cases it is often driven by storage. The expected number of concurrent queries and acceptable query latencies tend together with the type of storage used drive how much data each node can hold. This will vary by tier and you most likely need to test with your data and hardware in order to find out what works for your particular use case.

---

<div class="post-metadata">

**Author:** ![dsagent](https://avatars.discourse-cdn.com/v4/letter/d/d2c977/32.png) [@dsagent](https://discuss.elastic.co/u/dsagent)\
**Post date:** [October 7, 2024, 5:52pm UTC](https://discuss.elastic.co/t/what-are-the-requirements/368279/38 "2024-10-07T17:52:49Z")

</div>

What is the maximum amount of space that one node can afford ?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [October 7, 2024, 6:38pm UTC](https://discuss.elastic.co/t/what-are-the-requirements/368279/39 "2024-10-07T18:38:04Z")

</div>

It will depend on the specification of the node, the type of storage used and whether it performs indexing or not. The more data you put on a node, the slower queries will generally be. This is why you need to test it with your data, hardware and requirements around query latencies.

---

<div class="post-metadata">

**Author:** ![dsagent](https://avatars.discourse-cdn.com/v4/letter/d/d2c977/32.png) [@dsagent](https://discuss.elastic.co/u/dsagent)\
**Post date:** [October 7, 2024, 6:49pm UTC](https://discuss.elastic.co/t/what-are-the-requirements/368279/40 "2024-10-07T18:49:06Z")

</div>

> [@Christian\_Dahlqvist](#):
>
> It will depend on the specification of the node, the type of storage used and whether it performs indexing or not. The more data you put on a node, the slower queries will generally be. This is why you need to test it with your data, hardware and requirements around query latencies.

ok  
thank you

[Previous page](https://discuss.elastic.co/t/what-are-the-requirements/368279.md?page=1)

[Next page](https://discuss.elastic.co/t/what-are-the-requirements/368279.md?page=3)
