Searchable snapshots

[Morez (Mohammadreza Sahelgozin)]

As mentioned in the Elasticsearch documentation, searchable snapshot is created typically by ILM, when index reaches to the cold or frozen data tier.

Some times ago, it was mentioned that frozen data tier is deprecated in new versions (I can not find an active link about that nowadays as the links might be updated to introduce new frozen data tier)

Somewhere else, it is said that we have new cold and frozen data tiers from version 7.10 (beta version for searchable snapshot and new cold data tier) or higher versions.

I can understand the difference between deprecated frozen phase of ILM and new frozen data tier that matches to searchable snapshots as described in this link.

I also understand theory of searchable snapshot and the difference between using it in cold phase (which restores a complete version of snapshot on the cold data node) and in frozen phase (which restores data from snapshot partially based on the query and caches them for future similar queries). But I want to use one of these two scenario in practice (using searchable snapshot in cold phase or using it in frozen phase). However, I have problems with both of the scenarios:

1- When using searchable snapshot with cold phase, after reaching to the delete phase, when index in cold phase is going to be deleted, its snapshot also is being deleted. I want to know if there is a way to keep the snapshot for a longer period of time after index deletion in cold phase and manage its retention by other tools (e.g. SLM).

2- Based on the claims in the documentation about searchable snapshots in frozen data tier, I prefer to choose this scenario in comparison to using it in cold phase; having a hot data node, a warm data node, a frozen data node with searchable snapshot (which let me to keep data in this phase for a longer time). But I could not find any options or tools in Kibana to let me set such a scenario. I am not sure if it depends on the version I am using (7.12). Would you please tell me how to set practically an ILM with searchable snapshot in frozen data, step by step, or share a document which shows that?

[Morez (Mohammadreza Sahelgozin)]

Can anybody help me?

[leandrojmp (Leandro Pereira)]

I would suggest you to open a ticket with Elastic support to get a faster and better answer.

Searchable snapshots are only available with an Enterprise License and I think that there isn't many people in this forum that are using Searchable snapshots.

[stephenb (Stephen Brown)]

Hi @Morez welcome to the community.

In short Yes

Hot / Warm / Frozen is a valid architecture.

You will require an Enterprise License.

With Self Managed there will be some significant configuration that you will need to get correct.

I believe Frozen enabled by searchable snapshots is technically production in 7.13 but there was some fixes along the way I would probably use 7.15+.

ILM policies fully support Frozen / searchable snapshots.

You might find Elastic Cloud an easier way to test out your desired configuration.

[Morez (Mohammadreza Sahelgozin)]

Thank you so much for your reply.

What about my first question? do you know if it is possible to keep searchable snapshot of a cold index after reaching to ILM delete phase which commonly cause deleting both the index from data node and its searchable snapshot from snapshot repository?

Another question I have: is there any links to see which ES features are available in which versions?

[stephenb (Stephen Brown)]

@Morez

What about my first question? do you know if it is possible to keep searchable snapshot of a cold index after reaching to ILM delete phase which cause deleting of both index from data node and its searchable snapshot from snapshot repository?

I'm not sure I understand, but in short ILM allows deleting or retaining the searchable snapshot as part of the "delete phase", that is configurable.

You can see here that is a configurable option.

You can find subscriptions tiers here for self managed.. cold and Frozen backed by searchable snapshots are Enterprise tier.

Subscriptions | Elastic Stack Products & Support | Elastic

[Morez (Mohammadreza Sahelgozin)]

Thank you @stephenb

I was seeking exactly for the configuration you mentioned ( delete_searchable_snapshot )

What I understood is that I can not use (frozen data tier + searchable snapshot) as it is available from version 7.13 and later (my version: 7.12)
I can choose (cold data tier + searchable snapshot) scenario. My problem with this scenario was that I could not find delete_searchable_snapshot config and my snapshots used to be deleted when their indices were deleted. I will use and test it to see if it covers my needs.

Thank you again for your attention and kind help