Archive and purge

Elastic Stack Elasticsearch
1

Hi
I have a requirement where i need to delete old records from Elasticsearch
Index . Before i delete them i want to archive them to a different location
. I see that there are tools for exporting data (elasticsearch-knapsack)
and deleting index (curator) . Is there a tool which does following

  1. Archive records based on query from an Index
  2. Deletes above records from Index after successful Archive operation.

Your suggestions will be greatly appreciated .

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a0069343-938c-4858-b20d-0f62b69992ad%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

Yes, with the knapsack plugin, you can specify a query, and the result set
can be archived into tar.gz or zip archive. Knapsack has no feature for
automatic deletion of documents since the operation may lead to unexpected
data loss. Deletion should be done from shell after archiving has
succeeded, when an external validation of the created archive succeeds
(e.g. counting the number of entries in the archive). Knapsack may fail if
disk space is full or file write privilege is insufficient, so it is
essential to validate backups before proceeding. Best tool from shell to
kill documents is curl, either by index delete or delete by query command.

Jörg

On Wed, May 7, 2014 at 4:43 PM, Vijay Dodla vijay.remedy@gmail.com wrote:

Hi
I have a requirement where i need to delete old records from Elasticsearch
Index . Before i delete them i want to archive them to a different location
. I see that there are tools for exporting data (elasticsearch-knapsack)
and deleting index (curator) . Is there a tool which does following

  1. Archive records based on query from an Index
  2. Deletes above records from Index after successful Archive operation.

Your suggestions will be greatly appreciated .

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/a0069343-938c-4858-b20d-0f62b69992ad%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/a0069343-938c-4858-b20d-0f62b69992ad%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHnK%2BPg_JPfHJS_xtqm0%3D48trEWEX-MuJfgcVj-VwS-kg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

Thanks Jorg for the response .

On Wednesday, May 7, 2014 10:16:50 AM UTC-5, Jörg Prante wrote:

Yes, with the knapsack plugin, you can specify a query, and the result set
can be archived into tar.gz or zip archive. Knapsack has no feature for
automatic deletion of documents since the operation may lead to unexpected
data loss. Deletion should be done from shell after archiving has
succeeded, when an external validation of the created archive succeeds
(e.g. counting the number of entries in the archive). Knapsack may fail if
disk space is full or file write privilege is insufficient, so it is
essential to validate backups before proceeding. Best tool from shell to
kill documents is curl, either by index delete or delete by query command.

Jörg

On Wed, May 7, 2014 at 4:43 PM, Vijay Dodla <vijay....@gmail.com<javascript:>

wrote:

Hi
I have a requirement where i need to delete old records from
Elasticsearch Index . Before i delete them i want to archive them to a
different location . I see that there are tools for exporting data
(elasticsearch-knapsack) and deleting index (curator) . Is there a tool
which does following

  1. Archive records based on query from an Index
  2. Deletes above records from Index after successful Archive operation.

Your suggestions will be greatly appreciated .

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/a0069343-938c-4858-b20d-0f62b69992ad%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/a0069343-938c-4858-b20d-0f62b69992ad%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/30bf21cf-bd9b-4a78-b21c-0fef91fda5bb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.