# Elastic Search Data format conversion

**URL:** <https://discuss.elastic.co/t/elastic-search-data-format-conversion/154603>\
**Category:** Elasticsearch\
**Created:** [October 30, 2018, 9:54am UTC](https://discuss.elastic.co/t/elastic-search-data-format-conversion/154603 "2018-10-30T09:54:44Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![abhinav.dwivedi](https://avatars.discourse-cdn.com/v4/letter/a/f9ae1b/32.png) [@abhinav.dwivedi](https://discuss.elastic.co/u/abhinav.dwivedi)\
**Post date:** [October 30, 2018, 9:54am UTC](https://discuss.elastic.co/t/elastic-search-data-format-conversion/154603/1 "2018-10-30T09:54:44Z")

</div>

Team

I have 1 TB Elastic search data across 4 node cluster is there a way we can convert that data into CSV or Json?

Regards  
Abhinav D

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [October 30, 2018, 11:03am UTC](https://discuss.elastic.co/t/elastic-search-data-format-conversion/154603/2 "2018-10-30T11:03:57Z")

</div>

You can probably use logstash for this.

---

<div class="post-metadata">

**Author:** ![abhinav.dwivedi](https://avatars.discourse-cdn.com/v4/letter/a/f9ae1b/32.png) [@abhinav.dwivedi](https://discuss.elastic.co/u/abhinav.dwivedi)\
**Post date:** [October 30, 2018, 12:07pm UTC](https://discuss.elastic.co/t/elastic-search-data-format-conversion/154603/3 "2018-10-30T12:07:04Z")

</div>

hi dadoonet

i have 5 Grok filter in logstash . Could you please help me how can i do that is there any document for that?

Regards  
Abhinav D

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [October 30, 2018, 1:24pm UTC](https://discuss.elastic.co/t/elastic-search-data-format-conversion/154603/4 "2018-10-30T13:24:32Z")

</div>

I don't see the relationship between 5 grok filters and the initial question you asked.

You want to export data from Elasticsearch to a JSON file, right?  
This is not related to any filter.

---

<div class="post-metadata">

**Author:** ![abhinav.dwivedi](https://avatars.discourse-cdn.com/v4/letter/a/f9ae1b/32.png) [@abhinav.dwivedi](https://discuss.elastic.co/u/abhinav.dwivedi)\
**Post date:** [October 30, 2018, 3:53pm UTC](https://discuss.elastic.co/t/elastic-search-data-format-conversion/154603/5 "2018-10-30T15:53:50Z")

</div>

Yes we have all data 1.1 TB in either single json or multiple Json Or CSV .

Its possible?

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [October 30, 2018, 7:14pm UTC](https://discuss.elastic.co/t/elastic-search-data-format-conversion/154603/6 "2018-10-30T19:14:19Z")

</div>

Yes probably with logstash with an elasticsearch input plugin and a file output plugin ?

---

<div class="post-metadata">

**Author:** ![abhinav.dwivedi](https://avatars.discourse-cdn.com/v4/letter/a/f9ae1b/32.png) [@abhinav.dwivedi](https://discuss.elastic.co/u/abhinav.dwivedi)\
**Post date:** [November 5, 2018, 6:41am UTC](https://discuss.elastic.co/t/elastic-search-data-format-conversion/154603/7 "2018-11-05T06:41:36Z")

</div>

Hey

How can i know the data type[like : json,csv etc] for ES snapshot ?

Regards  
Abhinav

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [November 5, 2018, 10:03am UTC](https://discuss.elastic.co/t/elastic-search-data-format-conversion/154603/8 "2018-11-05T10:03:05Z")

</div>

If you mean the snapshot and restore feature, it's nothing like that. Let's say it's a binary format.

---

<div class="post-metadata">

**Author:** ![abhinav.dwivedi](https://avatars.discourse-cdn.com/v4/letter/a/f9ae1b/32.png) [@abhinav.dwivedi](https://discuss.elastic.co/u/abhinav.dwivedi)\
**Post date:** [November 5, 2018, 10:44am UTC](https://discuss.elastic.co/t/elastic-search-data-format-conversion/154603/9 "2018-11-05T10:44:44Z")

</div>

Hi Dadoonet

Actually, the problem is we have all the data in ES right now as I already mention. Around 1.1 TB data.

Logstash doesn't have much data its just filter out and pushes the data to ES.

So what i want to convert that 1.1 TB data in Json or csv . So i can put that JSON and CSV in S3 bucket and use one aws tool athena to query the data.

And clear my 1.1 TB and use ELK cluster for only one month data .

Regards  
Abhinav

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [November 5, 2018, 11:08am UTC](https://discuss.elastic.co/t/elastic-search-data-format-conversion/154603/10 "2018-11-05T11:08:22Z")

</div>

Use logstash to read from elasticsearch and write to S3.

---

<div class="post-metadata">

**Author:** ![abhinav.dwivedi](https://avatars.discourse-cdn.com/v4/letter/a/f9ae1b/32.png) [@abhinav.dwivedi](https://discuss.elastic.co/u/abhinav.dwivedi)\
**Post date:** [November 6, 2018, 9:11am UTC](https://discuss.elastic.co/t/elastic-search-data-format-conversion/154603/11 "2018-11-06T09:11:42Z")

</div>

hi

is there any document available for that ?

Regards  
Abhinav D

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 4, 2018, 9:11am UTC](https://discuss.elastic.co/t/elastic-search-data-format-conversion/154603/12 "2018-12-04T09:11:42Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
