Elasticsearch plugin - is there a way to keep it forever listening

Elastic Stack Logstash
1

Good morning,
At the moment, I have a query and it runs against the Logstash-* index but once it's done reading from today's date it closes the connection.

Is there a way to keep it always running so it will pick up any new logs? I tried adding stdin plugin to keep the script running, but after the Elasticsearch has done it's reading, it still closes the Elasticsearch part of the connection.

Please could someone help? Thank you for your time
Kind regards,
Chris

2

If you want to continuously monitor logs as they arrive it seems like a better idea to have them sent to a message broker like Redis or RabbitMQ.

3

Hi Magnus,

Thanks for your response, I have a fairly unique project where I have to pull the data from an Elasticsearch cluster rather than it getting pushed to a datasource. Not by choice unfortunately. I have a Logstash instance that pulls from a source Elasticsearch cluster and then pushes to syslog. Ideally live. at the moment I can do it so I can get yesterday's news. But Ideally, I would like what's happening right now.

If that's not possible then I'll have to stick with getting yesterday's logs. Unless the software above can relay from elasticsearch in a similar way?

Thanks again,