# Snapshots to S3, only meta files are being created in bucket

**URL:** https://discuss.elastic.co/t/snapshots-to-s3-only-meta-files-are-being-created-in-bucket/109994
**Category:** Elasticsearch
**Created:** [December 2, 2017, 2:16am UTC](https://discuss.elastic.co/t/snapshots-to-s3-only-meta-files-are-being-created-in-bucket/109994 "2017-12-02T02:16:52Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![mikebw](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mikebw/32/21766_2.png) [@mikebw](https://discuss.elastic.co/u/mikebw)
#### Post date: [December 2, 2017, 2:16am UTC](https://discuss.elastic.co/t/snapshots-to-s3-only-meta-files-are-being-created-in-bucket/109994/1 "2017-12-02T02:16:52Z")

</div>

I've setup an S3 as a repository for my cluster and when I create a snapshot, it seems to work, though in the bucket the only think I see created is the meta files.

For example if I do this request:

```auto
PUT https://mycreds@my_ip:8443/_snapshot/elk_s3_repository/test_snapshot
{
  "indices": "logstash-2017.11.24",
  "ignore_unavailable": true,
  "include_global_state": false
}

```

The snapshot created will eventually has this status:

```auto
{
    "snapshots": [
        {
            "snapshot": "test_snapshot",
            "uuid": "tqOQLw94TsmWcPuWoCUC_A",
            "version_id": 5040099,
            "version": "5.4.0",
            "indices": [
                "logstash-2017.11.24"
            ],
            "state": "SUCCESS",
            "start_time": "2017-12-02T01:33:01.854Z",
            "start_time_in_millis": 1512178381854,
            "end_time": "2017-12-02T01:33:17.189Z",
            "end_time_in_millis": 1512178397189,
            "duration_in_millis": 15335,
            "failures": [],
            "shards": {
                "total": 5,
                "failed": 0,
                "successful": 5
            }
        }
    ]
}

```

But when I look in the S3 bucket, the only files created are:

```auto
snap-tqOQLw94TsmWcPuWoCUC_A.dat . 13.5KB
snap-VI2VFbM5QYOUohR9ZQXCbQ.dat . 246B
meta-I2IjYzR-Tk6YEw-JhbQ-ZQ.dat 13.5K
snap-I2IjYzR-Tk6YEw-JhbQ-ZQ.dat . 246B

```

I would expect to see much larger files for a 59MB index.. What am I missing?

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [December 2, 2017, 3:18am UTC](https://discuss.elastic.co/t/snapshots-to-s3-only-meta-files-are-being-created-in-bucket/109994/2 "2017-12-02T03:18:36Z")

</div>

Normally you should also see a subdirectory named indices IIRC then. 0, 1, 2, 3, 4, then files in it.

---

<div class="post-metadata">

### Author: ![mikebw](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mikebw/32/21766_2.png) [@mikebw](https://discuss.elastic.co/u/mikebw)
#### Post date: [December 2, 2017, 5:33am UTC](https://discuss.elastic.co/t/snapshots-to-s3-only-meta-files-are-being-created-in-bucket/109994/3 "2017-12-02T05:33:15Z")

</div>

I see. I do not have that directory under my base directory.

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [December 2, 2017, 8:14am UTC](https://discuss.elastic.co/t/snapshots-to-s3-only-meta-files-are-being-created-in-bucket/109994/4 "2017-12-02T08:14:10Z")

</div>

Weird because it reported:

```auto
            "shards": {
                "total": 5,
                "failed": 0,
                "successful": 5
            }

```

Sorry to ask but are you sure you are looking at the right buckets?  
I prefer to double-check.

Can you do that again from scratch?

- delete the repository
- remove the S3 container
- create again the S3 container
- create the repository
- snapshot
- open [https://console.aws.amazon.com/s3/home](https://console.aws.amazon.com/s3/home) and list your files available in the container

Can you then share:

- all the commands you ran
- your elasticsearch logs

Also which version are you using?

---

<div class="post-metadata">

### Author: ![mikebw](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mikebw/32/21766_2.png) [@mikebw](https://discuss.elastic.co/u/mikebw)
#### Post date: [December 5, 2017, 12:54am UTC](https://discuss.elastic.co/t/snapshots-to-s3-only-meta-files-are-being-created-in-bucket/109994/5 "2017-12-05T00:54:56Z")

</div>

I recreated the S3 Repository by omitting the "base\_dir" option, and now all the sub folders and indices are showing up in the the root of the S3 bucket.

So before, files were showing up the base dir, but not all the files, and now ALL the files are showing up in the S3 bucket..

I will proceed for now without trying to specify a base\_dir.

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [December 5, 2017, 4:36am UTC](https://discuss.elastic.co/t/snapshots-to-s3-only-meta-files-are-being-created-in-bucket/109994/6 "2017-12-05T04:36:40Z")

</div>

What is the version you are using ?

---

<div class="post-metadata">

### Author: ![mikebw](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mikebw/32/21766_2.png) [@mikebw](https://discuss.elastic.co/u/mikebw)
#### Post date: [December 10, 2017, 6:56am UTC](https://discuss.elastic.co/t/snapshots-to-s3-only-meta-files-are-being-created-in-bucket/109994/7 "2017-12-10T06:56:34Z")

</div>

Elasticsearch 5.4.0

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 7, 2018, 6:56am UTC](https://discuss.elastic.co/t/snapshots-to-s3-only-meta-files-are-being-created-in-bucket/109994/8 "2018-01-07T06:56:53Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
