Create parquet files from an ES index snapshot?

(Shivkumar) #1

I am using S3 repository plugin to take snapshot of my ES Cluster to S3.
I have TB's of data on S3 ( roughly 5TB ) , I want to convert these snapshots to parquet files.
Is it possible to do so ?
Is it possible to configure ES itself, so that the snapshots are stored in parquet files format ?

(David Pilato) #2

No it's not.

(Shivkumar) #3

Is it possible to convert these snapshots to parquet files ?

(David Pilato) #4


But you can call the scroll API and get all the documents and do whatever needed transformation.
May be with logstash?

(Shivkumar) #5

Thank you

(system) #6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.