# No automatic logstash shutdown on Elasticsearch Input Plugin **URL:** \ **Category:** Logstash\ **Created:** [April 20, 2018, 1:45pm UTC](https://discuss.elastic.co/t/no-automatic-logstash-shutdown-on-elasticsearch-input-plugin/128887 "2018-04-20T13:45:21Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![sbienert](https://avatars.discourse-cdn.com/v4/letter/s/a3d4f5/32.png) [@sbienert](https://discuss.elastic.co/u/sbienert)\ **Post date:** [April 20, 2018, 1:45pm UTC](https://discuss.elastic.co/t/no-automatic-logstash-shutdown-on-elasticsearch-input-plugin/128887/1 "2018-04-20T13:45:22Z")
I have an index X which continuosly receives maschine logs. I need to reindex the data constantly to index Y which is connected to Kibana. I have a Elasticsearch input, filter and Elasticsearch output. The reindexing works fine but after all documents have been reindexed, Logstash automatically shuts down because probably one of the elasticsearch plugins sends out a shut down command. However, index X still receives logs which are then not stashed into index Y and in consequence cannot be seen in Kibana. I don't think it's a problem of my configuration, it's just the way the Elasticsearch plugin is programmed. I know a possible solution would be an exec plugin which starts the pipeline again and again in a given interval. But since the pipeline is starting and shutting down every time this would be very ugly and not ressource friendly. Is there any way to keep the pipeline with an Elasticsearch Input alive and not shutting it down automatically? Thanks in advance ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [April 22, 2018, 8:22pm UTC](https://discuss.elastic.co/t/no-automatic-logstash-shutdown-on-elasticsearch-input-plugin/128887/2 "2018-04-22T20:22:48Z")
Yes, nowadays there's a schedule option to make the ES query at certain intervals. ---
**Author:** ![sbienert](https://avatars.discourse-cdn.com/v4/letter/s/a3d4f5/32.png) [@sbienert](https://discuss.elastic.co/u/sbienert)\ **Post date:** [April 25, 2018, 6:58am UTC](https://discuss.elastic.co/t/no-automatic-logstash-shutdown-on-elasticsearch-input-plugin/128887/3 "2018-04-25T06:58:12Z")
Thanks for your answer. But what do you mean by "query at certain intervals"? Like Logstash quitting and starting again what I do not want or that the Elasticsearch Input keeps on running without shutting down (when did not received a sigterm) what I do want? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [April 25, 2018, 7:11am UTC](https://discuss.elastic.co/t/no-automatic-logstash-shutdown-on-elasticsearch-input-plugin/128887/4 "2018-04-25T07:11:38Z")
> Like Logstash quitting and starting again what I do not want No. > or that the Elasticsearch Input keeps on running without shutting down (when did not received a sigterm) what I do want? Yes. ---
**Author:** ![sbienert](https://avatars.discourse-cdn.com/v4/letter/s/a3d4f5/32.png) [@sbienert](https://discuss.elastic.co/u/sbienert)\ **Post date:** [April 25, 2018, 8:19am UTC](https://discuss.elastic.co/t/no-automatic-logstash-shutdown-on-elasticsearch-input-plugin/128887/5 "2018-04-25T08:19:49Z")
Okay so what is the trick? From my understanding the Elasticsearch Input quits when queried all documents automatically from default. ---
**Author:** ![sbienert](https://avatars.discourse-cdn.com/v4/letter/s/a3d4f5/32.png) [@sbienert](https://discuss.elastic.co/u/sbienert)\ **Post date:** [April 25, 2018, 8:22am UTC](https://discuss.elastic.co/t/no-automatic-logstash-shutdown-on-elasticsearch-input-plugin/128887/6 "2018-04-25T08:22:44Z")
Ah I found it. Thanks ---
**Author:** ![sbienert](https://avatars.discourse-cdn.com/v4/letter/s/a3d4f5/32.png) [@sbienert](https://discuss.elastic.co/u/sbienert)\ **Post date:** [April 27, 2018, 1:30pm UTC](https://discuss.elastic.co/t/no-automatic-logstash-shutdown-on-elasticsearch-input-plugin/128887/7 "2018-04-27T13:30:05Z")
@magnusbaeck Is the only option to query all of the documents again with the schedule option? Cause this takes really long with a lot of documents and is not ressource friendly. I want elasticsearch input plugin only to watch out for new documents. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [April 27, 2018, 5:18pm UTC](https://discuss.elastic.co/t/no-automatic-logstash-shutdown-on-elasticsearch-input-plugin/128887/8 "2018-04-27T17:18:52Z")
How would it know which documents are new? What sends the data to index X? Can you insert yourself there and fork the data stream so it's sent to both index X and index Y? ---
**Author:** ![sbienert](https://avatars.discourse-cdn.com/v4/letter/s/a3d4f5/32.png) [@sbienert](https://discuss.elastic.co/u/sbienert)\ **Post date:** [May 2, 2018, 9:14am UTC](https://discuss.elastic.co/t/no-automatic-logstash-shutdown-on-elasticsearch-input-plugin/128887/9 "2018-05-02T09:14:36Z")
> [@magnusbaeck](#): > > How would it know which documents are new? I do not know to be honest. How does Kibana watch out for new documents? Does it query the entire index again and again? > [@magnusbaeck](#): > > What sends the data to index X? Can you insert yourself there and fork the data stream so it's sent to both index X and index Y? This is difficult and I need to reindex anyway because otherwise most fields cannot be parsed correctly by Kibana. Changing the logging mechanism on the maschine side is not possible. Thank you. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [May 2, 2018, 12:21pm UTC](https://discuss.elastic.co/t/no-automatic-logstash-shutdown-on-elasticsearch-input-plugin/128887/10 "2018-05-02T12:21:03Z")
> I do not know to be honest. How does Kibana watch out for new documents? Does it query the entire index again and again? Yes, Kibana doesn't make incremental queries. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 30, 2018, 12:21pm UTC](https://discuss.elastic.co/t/no-automatic-logstash-shutdown-on-elasticsearch-input-plugin/128887/11 "2018-05-30T12:21:13Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.