# Create parquet files from an ES index snapshot?

**URL:** <https://discuss.elastic.co/t/create-parquet-files-from-an-es-index-snapshot/121326>\
**Category:** Elasticsearch\
**Created:** [February 24, 2018, 8:10am UTC](https://discuss.elastic.co/t/create-parquet-files-from-an-es-index-snapshot/121326 "2018-02-24T08:10:36Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![shivkumar](https://avatars.discourse-cdn.com/v4/letter/s/8797f3/32.png) [@shivkumar](https://discuss.elastic.co/u/shivkumar)\
**Post date:** [February 24, 2018, 8:10am UTC](https://discuss.elastic.co/t/create-parquet-files-from-an-es-index-snapshot/121326/1 "2018-02-24T08:10:36Z")

</div>

I am using S3 repository plugin to take snapshot of my ES Cluster to S3.  
I have TB's of data on S3 ( roughly 5TB ) , I want to convert these snapshots to parquet files.  
Is it possible to do so ?  
Is it possible to configure ES itself, so that the snapshots are stored in parquet files format ?

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [February 24, 2018, 8:21am UTC](https://discuss.elastic.co/t/create-parquet-files-from-an-es-index-snapshot/121326/2 "2018-02-24T08:21:50Z")

</div>

> [@shivkumar](#):
>
> Is it possible to configure ES itself, so that the snapshots are stored in parquet files format ?

No it's not.

---

<div class="post-metadata">

**Author:** ![shivkumar](https://avatars.discourse-cdn.com/v4/letter/s/8797f3/32.png) [@shivkumar](https://discuss.elastic.co/u/shivkumar)\
**Post date:** [February 24, 2018, 8:54am UTC](https://discuss.elastic.co/t/create-parquet-files-from-an-es-index-snapshot/121326/3 "2018-02-24T08:54:24Z")

</div>

Is it possible to convert these snapshots to parquet files ?

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [February 24, 2018, 9:06am UTC](https://discuss.elastic.co/t/create-parquet-files-from-an-es-index-snapshot/121326/4 "2018-02-24T09:06:03Z")

</div>

No.

But you can call the scroll API and get all the documents and do whatever needed transformation.  
May be with logstash?

---

<div class="post-metadata">

**Author:** ![shivkumar](https://avatars.discourse-cdn.com/v4/letter/s/8797f3/32.png) [@shivkumar](https://discuss.elastic.co/u/shivkumar)\
**Post date:** [February 24, 2018, 9:10am UTC](https://discuss.elastic.co/t/create-parquet-files-from-an-es-index-snapshot/121326/5 "2018-02-24T09:10:33Z")

</div>

Thank you

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 24, 2018, 9:11am UTC](https://discuss.elastic.co/t/create-parquet-files-from-an-es-index-snapshot/121326/6 "2018-03-24T09:11:00Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.