# How to "tail" an ElasticSearch index in LogStash

**URL:** <https://discuss.elastic.co/t/how-to-tail-an-elasticsearch-index-in-logstash/171488>\
**Category:** Logstash\
**Created:** [March 8, 2019, 11:56am UTC](https://discuss.elastic.co/t/how-to-tail-an-elasticsearch-index-in-logstash/171488 "2019-03-08T11:56:54Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![cawoodm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cawoodm/32/14083_2.png) [@cawoodm](https://discuss.elastic.co/u/cawoodm)\
**Post date:** [March 8, 2019, 11:56am UTC](https://discuss.elastic.co/t/how-to-tail-an-elasticsearch-index-in-logstash/171488/1 "2019-03-08T11:56:54Z")

</div>

We have all our logfile entries in a `logs-*` index and would like to tail it, choosing some special/error entries to push into another index `issues-*`.

- How can we query ElasticSearch for all logs.\* since time X where X is the last time we queried?

I imagine we'd need to persist the time we last queried in some way (file) and use this timestamp within an ES Query.

Of course there are workarounds: we could direct filebeat to send the logfile entries to `logs-*` and `issues-*`. We could also `clone` certain events we are sending to `logs-*` within logstash and send these to `issues-*`.

I just feel there should be a way to tail ES.

---

<div class="post-metadata">

**Author:** ![Wolfram\_Haussig](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wolfram_haussig/32/70528_2.png) [@Wolfram\_Haussig](https://discuss.elastic.co/u/Wolfram_Haussig)\
**Post date:** [March 8, 2019, 12:08pm UTC](https://discuss.elastic.co/t/how-to-tail-an-elasticsearch-index-in-logstash/171488/2 "2019-03-08T12:08:46Z")

</div>

Hi Marc,

I think you could use aliases too: [Aliases API | Elasticsearch Guide [8.11] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-aliases.html#filtered)

> ```
> POST /_aliases
> {
> "actions" : [
> {
> "add" : {
> "index" : "logs-*",
> "alias" : "issues",
> "filter" : { "term" : { "log.level" : "ERROR" } }
> }
> }
> ]
> }
> 
> ```

Then you could query this alias as:  
GET errorlog/\_search

Best regards  
Wolfram

---

<div class="post-metadata">

**Author:** ![cawoodm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cawoodm/32/14083_2.png) [@cawoodm](https://discuss.elastic.co/u/cawoodm)\
**Post date:** [March 8, 2019, 12:31pm UTC](https://discuss.elastic.co/t/how-to-tail-an-elasticsearch-index-in-logstash/171488/3 "2019-03-08T12:31:05Z")

</div>

That creates a view of the data I want but does not allow me to copy those docs into a separate index unless I do it daily/hourly for the previous day/hour. I really want to copy documents once and once only into a different index.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 5, 2019, 12:34pm UTC](https://discuss.elastic.co/t/how-to-tail-an-elasticsearch-index-in-logstash/171488/4 "2019-04-05T12:34:16Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.