# Best practices for automated incremental backups **URL:** \ **Category:** Elasticsearch\ **Created:** [March 19, 2018, 8:57am UTC](https://discuss.elastic.co/t/best-practices-for-automated-incremental-backups/124504 "2018-03-19T08:57:11Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![stevesimpson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stevesimpson/32/41437_2.png) [@stevesimpson](https://discuss.elastic.co/u/stevesimpson)\ **Post date:** [March 19, 2018, 8:57am UTC](https://discuss.elastic.co/t/best-practices-for-automated-incremental-backups/124504/1 "2018-03-19T08:57:11Z")
Hi, I've been looking into the backup and restore plan for our modest elasticsearch cluster. We are currently setup in a "Hot/Warm" architecture with curator moving old indexs from hot to warm. We currently have 2x Hot and 2x Warm (We will be expanding this as soon as budget allows). We use ZFS send/receive for most of our other services to perform snapshot and incremental backup ([https://docs.oracle.com/cd/E23824\_01/html/821-1448/gbciq.html](https://docs.oracle.com/cd/E23824_01/html/821-1448/gbciq.html)). We then ship the snapshots off to a remote backup server using ZFS send/receive ([https://docs.oracle.com/cd/E23824\_01/html/821-1448/gbchx.html#scrolltoc](https://docs.oracle.com/cd/E23824_01/html/821-1448/gbchx.html#scrolltoc)). Services like like MySQL, Postgresql etc all work fine using this method. After running a successful ZFS snapshot every hour and rolling back to a recent snapshot I'm unable to see any of the indexes, despite the data being back on the filesystem. ``` df -h Filesystem Size Used Avail Use% Mounted on elasticsearch 6.4T 128K 6.4T 1% /data/elasticsearch zfs rollback elasticsearch@2018-03-19_08:00:01 df -h Filesystem Size Used Avail Use% Mounted on elasticsearch 6.4T 47M 6.4T 1% /data/elasticsearch ``` Has anyone ever been able to successfully backup and restore ES using ZFS snapshots? Is there a better way to achieve an automated, incremental backup system within ES? We have no access to a HDFS FS however we could _possibly_ look at AWS S3 (This would be highly dependent on our data storage terms with our partners) Thanks! ---
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [March 19, 2018, 5:31pm UTC](https://discuss.elastic.co/t/best-practices-for-automated-incremental-backups/124504/2 "2018-03-19T17:31:56Z")
> [@stevesimpson](#): > > Has anyone ever been able to successfully backup and restore ES using ZFS snapshots? > Is there a better way to achieve an automated, incremental backup system within ES? You should not use filesystem backups to back up Elasticsearch indices. The appropriate way to ensure that both your _data_ and your _cluster state information_ are in sync is to use the Snapshot (and Restore) API. Performing a _filesystem_ snapshot is likely to result in slight deviations between what the cluster state _thinks_ is in the segment data, and what is _actually_ in the segment data. This happens because a filesystem snapshot could capture the cluster metadata at one point, and the segments could be different even a split second later. This will almost certainly result in corrupted index data at some point. Snapshots in Elasticsearch are at the _segment_ level, rather than at the _data_ level. An Elasticsearch Snapshot is an exact moment-in-time copy of the indices and their underlying segments. Incremental snapshots means that any _new_ segments will be copied to the repository when performing subsequent snapshots of the same index. Because segment merges happen continuously while new documents are added and removed, this means that _data_ (documents) might be re-copied, since they may have been merged into a new segment since the last snapshot. The only way to get automated, incremental backups is to use the Snapshot API on a scheduled basis. Since you're already using Curator, you can use [NFS](https://www.elastic.co/guide/en/elasticsearch/reference/6.2/modules-snapshots.html#_repositories), [S3](https://www.elastic.co/guide/en/elasticsearch/plugins/6.2/repository-s3.html), [Azure](https://www.elastic.co/guide/en/elasticsearch/plugins/6.2/repository-azure.html), [Google Cloud](https://www.elastic.co/guide/en/elasticsearch/plugins/6.2/repository-gcs.html), or [HDFS](https://www.elastic.co/guide/en/elasticsearch/plugins/6.2/repository-hdfs.html) as a snapshot repository, and have Curator take your periodic snapshots using the Snapshot API. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 16, 2018, 5:32pm UTC](https://discuss.elastic.co/t/best-practices-for-automated-incremental-backups/124504/3 "2018-04-16T17:32:01Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.