Snapshot and Restore in readable format

Hard to know for sure since you don't want to explain the use case fully and I don't know how much perfomance you're willing to sacrifice for getting the stuff back out of ES in a somewhat weird way.

Do consider that most of the time it makes a lot more sense to archive the raw events into readable formats at their point of entry compared to getting them back out from elasticsearch through search queries/full scan-dump/backups.
By that I mean something like logstash duplicating everything and shipping one copy into cold storage and the other copy into ES. Or whatever is receiving the events in the first place.
A nice example of that I have heard in recent history is a presenter at Elastic{ON} in SFO who explained how their logstash infrastructure duplicated everything to send 1 copy in AWS S3 and the other copy into ES.

Although I would strongly advise against re-extracting everything from ES the way you seem to want to do it, technically, it looks like you're looking for a tool I sometime use:

Martin

2 Likes