How to dump last 15 minutes of logstash data from elasticsearch?

Elastic Stack Elasticsearch
1

Greetings all,

I'm a novice elasticsearch user (I've recently begun monitoring a cluster
with logstash, elasticsearch and kibana. I'd like to be able, via cron, to
dump the last 15 minutes of logging data received to an external filesystem
(which will be pruned to only keep the last week or so of dumps). I'd then
like to import these files into a clean ES environment and, thereby, have
essentially the last week from the original ES environment. Is this
possible?

Thanks much,
NA

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/9a4e9f43-9d4f-491a-8748-452e0305167a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

Hey,

you could use a scroll request for this task with a specific query. See

As this simply returns JSON, you could use another script to import it into
another elasticearch environment. If you need some sort of backup
functionality, the snapshot/restore API might be interesting as well, see

--Alex

On Sat, Apr 12, 2014 at 7:18 PM, nyaqua botemout@gmail.com wrote:

Greetings all,

I'm a novice elasticsearch user (I've recently begun monitoring a cluster
with logstash, elasticsearch and kibana. I'd like to be able, via cron, to
dump the last 15 minutes of logging data received to an external filesystem
(which will be pruned to only keep the last week or so of dumps). I'd then
like to import these files into a clean ES environment and, thereby, have
essentially the last week from the original ES environment. Is this
possible?

Thanks much,
NA

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/9a4e9f43-9d4f-491a-8748-452e0305167a%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/9a4e9f43-9d4f-491a-8748-452e0305167a%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGCwEM9HzpUOdoL-c_7G9ptwfR0TPNVTyNwNiHsORiuHhn%3D%2BcA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

Thanks much, Alexander; I'll have a look.

On Monday, April 21, 2014 8:23:17 AM UTC-4, Alexander Reelsen wrote:

Hey,

you could use a scroll request for this task with a specific query. See
Elasticsearch Platform — Find real-time answers at scale | Elastic

As this simply returns JSON, you could use another script to import it
into another elasticearch environment. If you need some sort of backup
functionality, the snapshot/restore API might be interesting as well, see
Elasticsearch Platform — Find real-time answers at scale | Elastic

--Alex

On Sat, Apr 12, 2014 at 7:18 PM, nyaqua <bote...@gmail.com <javascript:>>wrote:

Greetings all,

I'm a novice elasticsearch user (I've recently begun monitoring a cluster
with logstash, elasticsearch and kibana. I'd like to be able, via cron, to
dump the last 15 minutes of logging data received to an external filesystem
(which will be pruned to only keep the last week or so of dumps). I'd then
like to import these files into a clean ES environment and, thereby, have
essentially the last week from the original ES environment. Is this
possible?

Thanks much,
NA

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/9a4e9f43-9d4f-491a-8748-452e0305167a%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/9a4e9f43-9d4f-491a-8748-452e0305167a%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/acaf5be9-4180-48e4-852c-335d79b9f7a6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.