# Elasticsearch input plugin not working **URL:** \ **Category:** Logstash\ **Created:** [May 2, 2017, 1:12pm UTC](https://discuss.elastic.co/t/elasticsearch-input-plugin-not-working/84266 "2017-05-02T13:12:35Z")\ **Posts on this page:** 16\ **Page:** 1
**Author:** ![Vittorio](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vittorio/32/18171_2.png) [@Vittorio](https://discuss.elastic.co/u/Vittorio)\ **Post date:** [May 2, 2017, 1:12pm UTC](https://discuss.elastic.co/t/elasticsearch-input-plugin-not-working/84266/1 "2017-05-02T13:12:35Z")
Hi all, I'm trying a simple pipeline where I use elasticsearch to get data and send them again to elastic but updated. For some reason the pipeline stops... Here the config I'm using: ``` input{ elasticsearch { hosts => "localhost" index => "dp_api-2017.04.28" docinfo => true query => '{"query": {"bool": {"must": [{"match": {"class": "DPAPINTERNAL"}},{"range": {"@timestamp": {"from": "now-10h","to": "now"}}}],"must_not": { "match": {"tags": "enriched" }}}}}' tags => ["to_enrich"] scroll => "2m" codec => "plain" } } filter{ if "to_enrich" in [tags]{ elasticsearch { #prendo i dpapi logs hosts => "localhost" index => "*" query_template => "/home/vittorio/Documents/offline-pipelines/conf.d-2/elastic-queries/matching-requestaw.json" fields => { "request" => "new_key" } add_tag => ["enriched", "output_splunk"] } } } output{ if "enriched" in [tags] { elasticsearch { hosts => "localhost" index => "%{index_name}-%{+YYYY.MM.dd}" document_id => "%{[@metadata][_id]}" action => "update" } } } ``` There's no error from logstash in debug mode.. ---
**Author:** ![Vittorio](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vittorio/32/18171_2.png) [@Vittorio](https://discuss.elastic.co/u/Vittorio)\ **Post date:** [May 2, 2017, 4:26pm UTC](https://discuss.elastic.co/t/elasticsearch-input-plugin-not-working/84266/2 "2017-05-02T16:26:30Z")
here is part (debug mode) of the shell result: ``` [2017-05-02T18:24:57,784][DEBUG][logstash.outputs.elasticsearch] Found existing Elasticsearch template. Skipping template management {:name=>"logstash"} [2017-05-02T18:24:57,785][INFO][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>[#]} [2017-05-02T18:24:57,790][INFO][logstash.filters.elasticsearch] New ElasticSearch filter {:hosts=>["localhost"]} [2017-05-02T18:24:57,855][INFO][logstash.pipeline] Starting pipeline {"id"=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>500} [2017-05-02T18:24:57,866][INFO][logstash.pipeline] Pipeline main started [2017-05-02T18:24:57,877][DEBUG][logstash.agent] Starting puma [2017-05-02T18:24:57,878][DEBUG][logstash.agent] Trying to start WebServer {:port=>9600} [2017-05-02T18:24:57,879][DEBUG][logstash.api.service] [api-service] start [2017-05-02T18:24:58,106][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=>9600} [2017-05-02T18:24:58,167][DEBUG][logstash.inputs.elasticsearch] closing {:plugin=>"LogStash::Inputs::Elasticsearch"} [2017-05-02T18:24:58,207][DEBUG][logstash.pipeline] Input plugins stopped! Will shutdown filter/output workers. [2017-05-02T18:24:58,271][DEBUG][logstash.pipeline] Pushing flush onto pipeline [2017-05-02T18:24:58,272][DEBUG][logstash.pipeline] Pushing shutdown {:thread=>"#"} [2017-05-02T18:24:58,272][DEBUG][logstash.pipeline] Pushing shutdown {:thread=>"#"} [2017-05-02T18:24:58,273][DEBUG][logstash.pipeline] Pushing shutdown {:thread=>"#"} [2017-05-02T18:24:58,273][DEBUG][logstash.pipeline] Pushing shutdown {:thread=>"#"} [2017-05-02T18:24:58,273][DEBUG][logstash.pipeline] Shutdown waiting for worker thread # [2017-05-02T18:24:58,300][DEBUG][logstash.pipeline] Shutdown waiting for worker thread # [2017-05-02T18:24:58,301][DEBUG][logstash.pipeline] Shutdown waiting for worker thread # [2017-05-02T18:24:58,301][DEBUG][logstash.pipeline] Shutdown waiting for worker thread # [2017-05-02T18:24:58,301][DEBUG][logstash.filters.elasticsearch] closing {:plugin=>"LogStash::Filters::Elasticsearch"} [2017-05-02T18:24:58,302][DEBUG][logstash.outputs.stdout] closing {:plugin=>"LogStash::Outputs::Stdout"} [2017-05-02T18:24:58,302][DEBUG][logstash.outputs.elasticsearch] closing {:plugin=>"LogStash::Outputs::ElasticSearch"} [2017-05-02T18:24:58,304][DEBUG][logstash.outputs.elasticsearch] Stopping sniffer [2017-05-02T18:24:58,304][DEBUG][logstash.outputs.elasticsearch] Stopping resurrectionist [2017-05-02T18:24:58,625][DEBUG][logstash.outputs.elasticsearch] Waiting for in use manticore connections [2017-05-02T18:24:58,626][DEBUG][logstash.outputs.elasticsearch] Closing adapter # [2017-05-02T18:24:58,627][DEBUG][logstash.pipeline] Pipeline main has been shutdown [2017-05-02T18:25:00,877][DEBUG][logstash.instrument.periodicpoller.os] PeriodicPoller: Stopping [2017-05-02T18:25:00,877][DEBUG][logstash.instrument.periodicpoller.jvm] PeriodicPoller: Stopping [2017-05-02T18:25:00,878][DEBUG][logstash.instrument.periodicpoller.persistentqueue] PeriodicPoller: Stopping [2017-05-02T18:25:00,880][WARN][logstash.agent] stopping pipeline {:id=>"main"} [2017-05-02T18:25:00,881][DEBUG][logstash.pipeline] Closing inputs [2017-05-02T18:25:00,881][DEBUG][logstash.inputs.elasticsearch] stopping {:plugin=>"LogStash::Inputs::Elasticsearch"} [2017-05-02T18:25:00,881][DEBUG][logstash.pipeline] Closed inputs ``` ---
**Author:** ![Vittorio](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vittorio/32/18171_2.png) [@Vittorio](https://discuss.elastic.co/u/Vittorio)\ **Post date:** [May 3, 2017, 7:33am UTC](https://discuss.elastic.co/t/elasticsearch-input-plugin-not-working/84266/3 "2017-05-03T07:33:12Z")
Please somebody take a look at this. ---
**Author:** ![Vittorio](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vittorio/32/18171_2.png) [@Vittorio](https://discuss.elastic.co/u/Vittorio)\ **Post date:** [May 3, 2017, 9:12am UTC](https://discuss.elastic.co/t/elasticsearch-input-plugin-not-working/84266/4 "2017-05-03T09:12:39Z")
is it normal maybe that when it finishes process the data it stops the pipeline? ---
**Author:** ![Baco](https://avatars.discourse-cdn.com/v4/letter/b/94ad74/32.png) [@Baco](https://discuss.elastic.co/u/Baco)\ **Post date:** [May 3, 2017, 9:47am UTC](https://discuss.elastic.co/t/elasticsearch-input-plugin-not-working/84266/5 "2017-05-03T09:47:40Z")
Hello Vittorio, Are you try to add port number to your elasticsearch in the output plugin? ---
**Author:** ![Vittorio](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vittorio/32/18171_2.png) [@Vittorio](https://discuss.elastic.co/u/Vittorio)\ **Post date:** [May 3, 2017, 9:54am UTC](https://discuss.elastic.co/t/elasticsearch-input-plugin-not-working/84266/6 "2017-05-03T09:54:35Z")
hello @Baco, thanks for your answer! I'm not sure I've understood your question, have you asked if I tried to put port number in the output or are you asking if I'm trying to add port number? If you refer to the first question, yes I tried to do `"localhost:9200"` ---
**Author:** ![Baco](https://avatars.discourse-cdn.com/v4/letter/b/94ad74/32.png) [@Baco](https://discuss.elastic.co/u/Baco)\ **Post date:** [May 3, 2017, 10:17am UTC](https://discuss.elastic.co/t/elasticsearch-input-plugin-not-working/84266/7 "2017-05-03T10:17:17Z")
Yes, my question was if you did "localhost:9200" Sorry, but I will continue to look the configuration. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [May 4, 2017, 5:31am UTC](https://discuss.elastic.co/t/elasticsearch-input-plugin-not-working/84266/8 "2017-05-04T05:31:06Z")
Is the elasticsearch input even supposed to keep Logstash alive? I suspect it's designed to fire off the query once, send the results down the pipeline, and then shut down Logstash. ---
**Author:** ![Vittorio](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vittorio/32/18171_2.png) [@Vittorio](https://discuss.elastic.co/u/Vittorio)\ **Post date:** [May 8, 2017, 7:13am UTC](https://discuss.elastic.co/t/elasticsearch-input-plugin-not-working/84266/9 "2017-05-08T07:13:49Z")
Hi @magnusbaeck thanks for the answer. I think you are right, that is strange though.... What about I want to query elasticsearch every 5m for example and execute the same pipeline again? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [May 8, 2017, 7:17am UTC](https://discuss.elastic.co/t/elasticsearch-input-plugin-not-working/84266/10 "2017-05-08T07:17:08Z")
Since the elasticsearch input lacks a schedule option (unlike the jdbc input) you can always run Logstash via cron. But yes, it would be more convenient if this was built in. ---
**Author:** ![Vittorio](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vittorio/32/18171_2.png) [@Vittorio](https://discuss.elastic.co/u/Vittorio)\ **Post date:** [May 8, 2017, 7:26am UTC](https://discuss.elastic.co/t/elasticsearch-input-plugin-not-working/84266/11 "2017-05-08T07:26:18Z")
How do I run logstash via cron? Using the jdbc input plugin? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [May 8, 2017, 7:28am UTC](https://discuss.elastic.co/t/elasticsearch-input-plugin-not-working/84266/12 "2017-05-08T07:28:29Z")
What part are you finding difficult? I don't have time to explain cron in general but if there are any Logstash-specific issues I can chip in. ---
**Author:** ![Vittorio](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vittorio/32/18171_2.png) [@Vittorio](https://discuss.elastic.co/u/Vittorio)\ **Post date:** [May 8, 2017, 7:31am UTC](https://discuss.elastic.co/t/elasticsearch-input-plugin-not-working/84266/13 "2017-05-08T07:31:15Z")
Thanks but i don't need you to eplain cron in general, but as you said, start logstash via cron is something new for me ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [May 8, 2017, 7:34am UTC](https://discuss.elastic.co/t/elasticsearch-input-plugin-not-working/84266/14 "2017-05-08T07:34:23Z")
Okay, but what's difficult about starting Logstash from cron? I'm sure you've started Logstash from a terminal before and I don't see why cron would be materially different. ---
**Author:** ![Vittorio](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vittorio/32/18171_2.png) [@Vittorio](https://discuss.elastic.co/u/Vittorio)\ **Post date:** [May 8, 2017, 7:38am UTC](https://discuss.elastic.co/t/elasticsearch-input-plugin-not-working/84266/15 "2017-05-08T07:38:20Z")
Oh I see, you mean setting up a cronjob... I was thinking something inside logstash for instance using some plugins ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 5, 2017, 7:49am UTC](https://discuss.elastic.co/t/elasticsearch-input-plugin-not-working/84266/16 "2017-06-05T07:49:25Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.