# Logstash's Pipeline has terminated **URL:** \ **Category:** Logstash\ **Created:** [February 28, 2018, 4:10am UTC](https://discuss.elastic.co/t/logstashs-pipeline-has-terminated/121772 "2018-02-28T04:10:29Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![alexus](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alexus/32/12696_2.png) [@alexus](https://discuss.elastic.co/u/alexus)\ **Post date:** [February 28, 2018, 4:10am UTC](https://discuss.elastic.co/t/logstashs-pipeline-has-terminated/121772/1 "2018-02-28T04:10:29Z")
I'm [Running Logstash on Docker](https://www.elastic.co/guide/en/logstash/current/docker.html), yet running into following issue: ``` logstash_1 | Sending Logstash's logs to /usr/share/logstash/logs which is now configured via log4j2.properties logstash_1 | [2018-02-28T04:02:06,811][INFO][logstash.modules.scaffold] Initializing module {:module_name=>"netflow", :directory=>"/usr/share/logstash/modules/netflow/configuration"} logstash_1 | [2018-02-28T04:02:06,833][INFO][logstash.modules.scaffold] Initializing module {:module_name=>"fb_apache", :directory=>"/usr/share/logstash/modules/fb_apache/configuration"} logstash_1 | [2018-02-28T04:02:07,991][INFO][logstash.modules.scaffold] Initializing module {:module_name=>"arcsight", :directory=>"/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/x-pack-6.2.1-java/modules/arcsight/configuration"} logstash_1 | [2018-02-28T04:02:08,214][INFO][logstash.setting.writabledirectory] Creating directory {:setting=>"path.queue", :path=>"/usr/share/logstash/data/queue"} logstash_1 | [2018-02-28T04:02:08,219][INFO][logstash.setting.writabledirectory] Creating directory {:setting=>"path.dead_letter_queue", :path=>"/usr/share/logstash/data/dead_letter_queue"} logstash_1 | [2018-02-28T04:02:08,877][INFO][logstash.agent] No persistent UUID file found. Generating new UUID {:uuid=>"880eedb5-4fea-4989-801c-01157e5891ef", :path=>"/usr/share/logstash/data/uuid"} logstash_1 | [2018-02-28T04:02:09,576][INFO][logstash.runner] Starting Logstash {"logstash.version"=>"6.2.1"} logstash_1 | [2018-02-28T04:02:10,281][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=>9600} logstash_1 | [2018-02-28T04:02:14,322][INFO][logstash.pipeline] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50} logstash_1 | [2018-02-28T04:02:15,218][INFO][logstash.pipeline] Pipeline started succesfully {:pipeline_id=>"main", :thread=>"#"} logstash_1 | [2018-02-28T04:02:15,381][INFO][logstash.agent] Pipelines running {:count=>1, :pipelines=>["main"]} logstash_1 | [2018-02-28T04:02:18,052][INFO][logstash.pipeline] Pipeline has terminated {:pipeline_id=>"main", :thread=>"#"} logstash_logstash_1 exited with code 0 # ``` I re-run it with `LOG_LEVEL=debug` and notices following error: ``` logstash_1 | [2018-02-28T04:03:29,398][DEBUG][logstash.instrument.periodicpoller.cgroup] Error, cannot retrieve cgroups information {:exception=>"Errno::ENOENT", :message=>"No such file or directory - /sys/fs/cgroup/cpuacct/docker/9f2a8ce98ade7aa55bf99896ba8e088d1b028db9fcd332a3d56737e322bb5a31/cpuacct.usage"} logstash_1 | [2018-02-28T04:03:29,471][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ParNew"} logstash_1 | [2018-02-28T04:03:29,481][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ConcurrentMarkSweep"} logstash_1 | [2018-02-28T04:03:29,543][INFO][logstash.pipeline] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>2, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50} logstash_1 | [2018-02-28T04:03:30,255][INFO][logstash.pipeline] Pipeline started succesfully {:pipeline_id=>"main", :thread=>"#"} logstash_1 | [2018-02-28T04:03:30,449][INFO][logstash.agent] Pipelines running {:count=>1, :pipelines=>["main"]} logstash_1 | [2018-02-28T04:03:33,195][DEBUG][logstash.inputs.elasticsearch] Closing {:plugin=>"LogStash::Inputs::Elasticsearch"} logstash_1 | [2018-02-28T04:03:33,239][DEBUG][logstash.pipeline] Pushing flush onto pipeline {:pipeline_id=>"main", :thread=>"#"} logstash_1 | [2018-02-28T04:03:33,246][DEBUG][logstash.pipeline] Shutting down filter/output workers {:pipeline_id=>"main", :thread=>"#"} logstash_1 | [2018-02-28T04:03:33,255][DEBUG][logstash.pipeline] Pushing shutdown {:pipeline_id=>"main", :thread=>"#worker0 run>"} logstash_1 | [2018-02-28T04:03:33,257][DEBUG][logstash.pipeline] Pushing shutdown {:pipeline_id=>"main", :thread=>"#worker1 run>"} logstash_1 | [2018-02-28T04:03:33,266][DEBUG][logstash.pipeline] Shutdown waiting for worker thread {:pipeline_id=>"main", :thread=>"#worker0 run>"} logstash_1 | [2018-02-28T04:03:33,436][DEBUG][logstash.pipeline] Shutdown waiting for worker thread {:pipeline_id=>"main", :thread=>"#worker1 dead>"} logstash_1 | [2018-02-28T04:03:33,442][DEBUG][logstash.filters.mutate] Closing {:plugin=>"LogStash::Filters::Mutate"} logstash_1 | [2018-02-28T04:03:33,455][DEBUG][logstash.outputs.csv] Closing {:plugin=>"LogStash::Outputs::CSV"} logstash_1 | [2018-02-28T04:03:33,465][DEBUG][logstash.outputs.csv] Close: closing files logstash_1 | [2018-02-28T04:03:33,490][INFO][logstash.pipeline] Pipeline has terminated {:pipeline_id=>"main", :thread=>"#"} logstash_1 | [2018-02-28T04:03:33,544][DEBUG][logstash.instrument.periodicpoller.os] Stopping logstash_1 | [2018-02-28T04:03:33,585][DEBUG][logstash.instrument.periodicpoller.jvm] Stopping logstash_1 | [2018-02-28T04:03:33,587][DEBUG][logstash.instrument.periodicpoller.persistentqueue] Stopping logstash_1 | [2018-02-28T04:03:33,596][DEBUG][logstash.instrument.periodicpoller.deadletterqueue] Stopping logstash_1 | [2018-02-28T04:03:33,710][DEBUG][logstash.agent] Shutting down all pipelines {:pipelines_count=>1} logstash_1 | [2018-02-28T04:03:33,718][DEBUG][logstash.agent] Converging pipelines state {:actions_count=>1} logstash_1 | [2018-02-28T04:03:33,720][DEBUG][logstash.agent] Executing action {:action=>LogStash::PipelineAction::Stop/pipeline_id:main} logstash_1 | [2018-02-28T04:03:33,744][DEBUG][logstash.pipeline] Stopping inputs {:pipeline_id=>"main", :thread=>"#"} logstash_1 | [2018-02-28T04:03:33,749][DEBUG][logstash.inputs.elasticsearch] Stopping {:plugin=>"LogStash::Inputs::Elasticsearch"} logstash_1 | [2018-02-28T04:03:33,762][DEBUG][logstash.pipeline] Stopped inputs {:pipeline_id=>"main", :thread=>"#"} logstash_1 | [2018-02-28T04:03:33,780][DEBUG][logstash.pipeline] Worker terminated {:pipeline_id=>"main", :thread=>"#worker0 dead>"} logstash_1 | [2018-02-28T04:03:33,785][DEBUG][logstash.pipeline] Worker terminated {:pipeline_id=>"main", :thread=>"#worker1 dead>"} logstash_logstash_1 exited with code 0 ``` Please advise. ---
**Author:** ![Makra](https://avatars.discourse-cdn.com/v4/letter/m/8491ac/32.png) [@Makra](https://discuss.elastic.co/u/Makra)\ **Post date:** [February 28, 2018, 5:16am UTC](https://discuss.elastic.co/t/logstashs-pipeline-has-terminated/121772/2 "2018-02-28T05:16:43Z")
First check the configuration of logstash config using the command like below. $ logstash --config.test\_and\_exit -f /etc/logstash/conf.d/test.conf ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [February 28, 2018, 7:46am UTC](https://discuss.elastic.co/t/logstashs-pipeline-has-terminated/121772/3 "2018-02-28T07:46:44Z")
Unless you've enabled scheduled execution I believe the elasticsearch input shuts down Logstash after it has run the configured query once. ---
**Author:** ![alexus](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alexus/32/12696_2.png) [@alexus](https://discuss.elastic.co/u/alexus)\ **Post date:** [February 28, 2018, 3:55pm UTC](https://discuss.elastic.co/t/logstashs-pipeline-has-terminated/121772/4 "2018-02-28T15:55:21Z")
lines such as: `Pipeline started succesfully` and/or `Pipelines running`, tells me config.test\_and\_exit would be OK. ---
**Author:** ![alexus](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alexus/32/12696_2.png) [@alexus](https://discuss.elastic.co/u/alexus)\ **Post date:** [February 28, 2018, 3:57pm UTC](https://discuss.elastic.co/t/logstashs-pipeline-has-terminated/121772/5 "2018-02-28T15:57:27Z")
there is no schedule execution... my output consists of csv and at the end, there is no .csv file generated... I ran same query through Kibana's Dev Tools and it gives desired results, but not through logstash( ---
**Author:** ![pjanzen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pjanzen/32/13756_2.png) [@pjanzen](https://discuss.elastic.co/u/pjanzen)\ **Post date:** [February 28, 2018, 4:17pm UTC](https://discuss.elastic.co/t/logstashs-pipeline-has-terminated/121772/6 "2018-02-28T16:17:38Z")
Can you post you the config you use and the logstash.yml file? ---
**Author:** ![alexus](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alexus/32/12696_2.png) [@alexus](https://discuss.elastic.co/u/alexus)\ **Post date:** [February 28, 2018, 4:32pm UTC](https://discuss.elastic.co/t/logstashs-pipeline-has-terminated/121772/7 "2018-02-28T16:32:47Z")
``` # cat pipeline/* input { elasticsearch { hosts => ["elasticsearch:9200"] index => "x*" password => "x" user => "x" query => ' { "query": { "bool": { "should": [ {"wildcard":{"x":"x*"}} ], "minimum_should_match": 1, "filter": [{ "exists": { "field": "x" } }, { "exists": { "field": "x" } }, { "exists": { "field": "x" } }, { "range": { "@timestamp": { "gte": "2017-07-01T00:00:00.000Z", "lte": "2018-01-01T23:59:59.999Z" } } }] } } } ' size => 500 scroll => "5m" } } filter { mutate { add_field => { "x" => "x" } } } output { csv { fields => ["x","_id","x","x","x","x","x","x","x","x","x","x","x","x","x","x","x","x","x","x","x","x","x","x","x","x","x","x","x","x","x","x","x","x","x","x","x","x","x"] path => "/tmp/x.csv" } } # ``` ---
**Author:** ![pjanzen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pjanzen/32/13756_2.png) [@pjanzen](https://discuss.elastic.co/u/pjanzen)\ **Post date:** [February 28, 2018, 5:55pm UTC](https://discuss.elastic.co/t/logstashs-pipeline-has-terminated/121772/8 "2018-02-28T17:55:48Z")
Looks easy enough.. Your sure you can connect to the ES instance? ---
**Author:** ![alexus](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alexus/32/12696_2.png) [@alexus](https://discuss.elastic.co/u/alexus)\ **Post date:** [February 28, 2018, 7:41pm UTC](https://discuss.elastic.co/t/logstashs-pipeline-has-terminated/121772/9 "2018-02-28T19:41:05Z")
I'm pretty sure, or at least I don't see any reason why not... I have few other pipeline configuration, and they connects to elasticsearch without any issues, the only thing I change is pipeline configuration (different query). ---
**Author:** ![alexus](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alexus/32/12696_2.png) [@alexus](https://discuss.elastic.co/u/alexus)\ **Post date:** [February 28, 2018, 10:11pm UTC](https://discuss.elastic.co/t/logstashs-pipeline-has-terminated/121772/10 "2018-02-28T22:11:07Z")
n/m, I figure out where I made a mistake and as soon as I correct it, it's started to work) ---
**Author:** ![elaair](https://avatars.discourse-cdn.com/v4/letter/e/6f9a4e/32.png) [@elaair](https://discuss.elastic.co/u/elaair)\ **Post date:** [March 6, 2018, 1:39am UTC](https://discuss.elastic.co/t/logstashs-pipeline-has-terminated/121772/11 "2018-03-06T01:39:34Z")
what was your mistake? ---
**Author:** ![alexus](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alexus/32/12696_2.png) [@alexus](https://discuss.elastic.co/u/alexus)\ **Post date:** [March 6, 2018, 2:18am UTC](https://discuss.elastic.co/t/logstashs-pipeline-has-terminated/121772/12 "2018-03-06T02:18:47Z")
I was playing with Security and lock myself to certain docs, and was trying to query to others... it was working, just no data since my query was returning nothing... hence nothing was written to the file... ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 3, 2018, 2:19am UTC](https://discuss.elastic.co/t/logstashs-pipeline-has-terminated/121772/13 "2018-04-03T02:19:28Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.