Create parquet files from an ES index snapshot?


(Shivkumar) #1

I am using S3 repository plugin to take snapshot of my ES Cluster to S3.
I have TB's of data on S3 ( roughly 5TB ) , I want to convert these snapshots to parquet files.
Is it possible to do so ?
Is it possible to configure ES itself, so that the snapshots are stored in parquet files format ?


(David Pilato) #2

No it's not.


(Shivkumar) #3

Is it possible to convert these snapshots to parquet files ?


(David Pilato) #4

No.

But you can call the scroll API and get all the documents and do whatever needed transformation.
May be with logstash?


(Shivkumar) #5

Thank you


(system) #6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.