How snapshot works and its impact

Elastic Stack Elasticsearch
1

Hi all,

I have multiple question

  1. How the snapshot module works ?
  2. How it knows where exactly to snapshot when using incremental snapshot ?
  3. Is it a simple search query ?
  4. The impact in term of cpu ?

Thank you all.

Nicolas.

2

A bit old but this should explain : https://www.elastic.co/blog/found-elasticsearch-snapshot-and-restore

3

Thanks David,

The link answers to the first two questions. For the third and the fourth ? Do you have an idea ?

Thanks.

4

Lucene is the underlying search engine and it only creates new files - it never updates them.
The activity of indexing flushes new mini-indexes out to disk as "segment files". Lucene background tasks merge small segments into larger ones as a form of compaction and then delete the newly redundant segments. You can see this process here [1].
Snapshot is only concerned with copying over the segment files that are created since the last time you ran a snapshot. No queries required - it's just syncing sets of files.

[1] http://blog.mikemccandless.com/2011/02/visualizing-lucenes-segment-merges.html

5

Thanks Mark,

Last question, does the snapshot could impact the search thread pool ?
All this questions, because I have created a cron that call the snapshot api each hour. And each hour I see the thread pool of the search requests reaching its limit to 1000...

For information I'm on 1.7 version !

Nicolas.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.