# Pipeline terminated {"pipeline.id"=\>".monitoring-logstash"} Logstash shutdown **URL:** \ **Category:** Logstash\ **Tags:** docker\ **Created:** [October 8, 2019, 5:53pm UTC](https://discuss.elastic.co/t/pipeline-terminated-pipeline-id-monitoring-logstash-logstash-shutdown/202709 "2019-10-08T17:53:41Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![happycampo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/happycampo/32/55494_2.png) [@happycampo](https://discuss.elastic.co/u/happycampo)\ **Post date:** [October 8, 2019, 5:53pm UTC](https://discuss.elastic.co/t/pipeline-terminated-pipeline-id-monitoring-logstash-logstash-shutdown/202709/1 "2019-10-08T17:53:41Z")
I finally got my elk stack running and working with mysql-connector and then I decided to resize my vm. Now logstash will not stay up. This was working before I'm pretty sure. Logstash.yml --- > ## Default Logstash configuration from Logstash base image. > > ## [https://github.com/elastic/logstash/blob/master/docker/data/logstash/config/logstash-full.yml](https://github.com/elastic/logstash/blob/master/docker/data/logstash/config/logstash-full.yml) > > # > > http.host: "0.0.0.0" > xpack.monitoring.elasticsearch.hosts: ["[http://elasticsearch:9200](http://elasticsearch:9200)" ] > path.config: /usr/share/logstash/pipeline > > ## X-Pack security credentials > > # > > xpack.monitoring.enabled: true > xpack.monitoring.elasticsearch.username: elastic > xpack.monitoring.elasticsearch.password: I tried commenting out the path.config as was suggested in another post but same issue. logstash.conf > input { > jdbc { > jdbc\_connection\_string =\> "jdbc:mysql://10.50.0.11:3306/ncryptedcloud" > jdbc\_user =\> "ncryptedcloud" > jdbc\_password =\> "db3Looking4Charlie@NCC" > jdbc\_driver\_class =\> "" > jdbc\_driver\_library =\> "" > jdbc\_default\_timezone =\> "UTC" > statement =\> "select \* from table" > } > } > > ## Add your filters / logstash plugins configuration here > > output { > elasticsearch { > hosts =\> "elasticsearch:9200" > user =\> "elastic" > password =\> "changeme" > index =\> "metadata-sql" > } > } docker-compose.xml > logstash: > build: > context: logstash/ > args: > ELK\_VERSION: $ELK\_VERSION > volumes: > - type: bind > source: ./logstash/config/logstash.yml > target: /usr/share/logstash/config/logstash.yml > read\_only: true > - type: bind > source: ./logstash/pipeline > target: /usr/share/logstash/pipeline > read\_only: true > - type: bind > source: ./mysql-connector-java-8.0.17 > target: /usr/share/mysql-connector-java-8.0.17 > ports: > - "5000:5000" > - "9600:9600" > environment: > LS\_JAVA\_OPTS: "-Xmx256m -Xms256m" > networks: > - elk > depends\_on: > - elasticsearch logstash/Dockerfile > ARG ELK\_VERSION > > # [GitHub - elastic/logstash-docker: Official Logstash Docker image](https://github.com/elastic/logstash-docker) > > FROM [docker.elastic.co/logstash/logstash:${ELK\_VERSION}](http://docker.elastic.co/logstash/logstash:$%7BELK_VERSION%7D) > ADD mysql-connector-java-8.0.17.jar /usr/share/logstash/logstash-core/lib/jars > > # Add your logstash plugins setup here > > # Example: RUN logstash-plugin install logstash-filter-json ---
**Author:** ![happycampo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/happycampo/32/55494_2.png) [@happycampo](https://discuss.elastic.co/u/happycampo)\ **Post date:** [October 8, 2019, 5:55pm UTC](https://discuss.elastic.co/t/pipeline-terminated-pipeline-id-monitoring-logstash-logstash-shutdown/202709/2 "2019-10-08T17:55:42Z")
HEre is the log: > logstash\_1 | [2019-10-08T17:35:49,355][INFO][logstash.monitoring.internalpipelinesource] Validated license for monitoring. Enabling monitoring pipeline. > logstash\_1 | [2019-10-08T17:35:50,930][INFO][org.reflections.Reflections] Reflections took 69 ms to scan 1 urls, producing 19 keys and 39 values > logstash\_1 | [2019-10-08T17:35:51,397][INFO][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=\>{:removed=\>, :added=\>[[http://elastic:xxxxxx@elasticsearch:9200/](http://elastic:xxxxxx@elasticsearch:9200/)]}} > logstash\_1 | [2019-10-08T17:35:51,455][WARN][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=\>"[http://elastic:xxxxxx@elasticsearch:9200/](http://elastic:xxxxxx@elasticsearch:9200/)"} > logstash\_1 | [2019-10-08T17:35:51,471][INFO][logstash.outputs.elasticsearch] ES Output version determined {:es\_version=\>7} > logstash\_1 | [2019-10-08T17:35:51,472][WARN][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document \_type {:es\_version=\>7} > logstash\_1 | [2019-10-08T17:35:51,493][INFO][logstash.outputs.elasticsearch] New Elasticsearch output {:class=\>"LogStash::Outputs::Elasticsearch", :hosts=\>["//elasticsearch:9200"]} > logstash\_1 | [2019-10-08T17:35:51,717][INFO][logstash.outputs.elasticsearch] Using default mapping template > logstash\_1 | [2019-10-08T17:35:51,747][WARN][org.logstash.instrument.metrics.gauge.LazyDelegatingGauge] A gauge metric of an unknown type (org.jruby.specialized.RubyArrayOneObject) has been create for key: cluster\_uuids. This may result in invalid serialization. It is recommended to log an issue to the responsible developer/development team. > logstash\_1 | [2019-10-08T17:35:51,753][INFO][logstash.javapipeline] Starting pipeline {:pipeline\_id=\>"main", "pipeline.workers"=\>8, "pipeline.batch.size"=\>125, "pipeline.batch.delay"=\>50, "pipeline.max\_inflight"=\>1000, :thread=\>"#\"} > logstash\_1 | [2019-10-08T17:35:51,790][INFO][logstash.outputs.elasticsearch] Attempting to install template {:manage\_template=\>{"index\_patterns"=\>"logstash-_", "version"=\>60001, "settings"=\>{"index.refresh\_interval"=\>"5s", "number\_of\_shards"=\>1}, "mappings"=\>{"dynamic\_templates"=\>[{"message\_field"=\>{"path\_match"=\>"message", "match\_mapping\_type"=\>"string", "mapping"=\>{"type"=\>"text", "norms"=\>false}}}, {"string\_fields"=\>{"match"=\>"_", "match\_mapping\_type"=\>"string", "mapping"=\>{"type"=\>"text", "norms"=\>false, "fields"=\>{"keyword"=\>{"type"=\>"keyword", "ignore\_above"=\>256}}}}}], "properties"=\>{"@timestamp"=\>{"type"=\>"date"}, "@version"=\>{"type"=\>"keyword"}, "geoip"=\>{"dynamic"=\>true, "properties"=\>{"ip"=\>{"type"=\>"ip"}, "location"=\>{"type"=\>"geo\_point"}, "latitude"=\>{"type"=\>"half\_float"}, "longitude"=\>{"type"=\>"half\_float"}}}}}}} > logstash\_1 | [2019-10-08T17:35:51,856][INFO][logstash.outputs.elasticsearch] Installing elasticsearch template to \_template/logstash > logstash\_1 | [2019-10-08T17:35:52,142][INFO][logstash.javapipeline] Pipeline started {"pipeline.id"=\>"main"} > logstash\_1 | [2019-10-08T17:35:52,315][INFO][logstash.agent] Pipelines running {:count=\>1, :running\_pipelines=\>[:main], :non\_running\_pipelines=\>} > logstash\_1 | [2019-10-08T17:35:53,473][INFO][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=\>{:removed=\>, :added=\>[[http://elastic:xxxxxx@elasticsearch:9200/](http://elastic:xxxxxx@elasticsearch:9200/)]}} > logstash\_1 | [2019-10-08T17:35:53,491][WARN][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=\>"[http://elastic:xxxxxx@elasticsearch:9200/](http://elastic:xxxxxx@elasticsearch:9200/)"} > logstash\_1 | [2019-10-08T17:35:53,520][INFO][logstash.outputs.elasticsearch] ES Output version determined {:es\_version=\>7} > logstash\_1 | [2019-10-08T17:35:53,520][WARN][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document \_type {:es\_version=\>7} > logstash\_1 | [2019-10-08T17:35:53,537][INFO][logstash.outputs.elasticsearch] New Elasticsearch output {:class=\>"LogStash::Outputs::Elasticsearch", :hosts=\>["[http://elasticsearch:9200](http://elasticsearch:9200)"]} > logstash\_1 | [2019-10-08T17:35:53,550][INFO][logstash.javapipeline] Starting pipeline {:pipeline\_id=\>".monitoring-logstash", "pipeline.workers"=\>1, "pipeline.batch.size"=\>2, "pipeline.batch.delay"=\>50, "pipeline.max\_inflight"=\>2, :thread=\>"#\"} > logstash\_1 | [2019-10-08T17:35:53,614][INFO][logstash.javapipeline] Pipeline started {"pipeline.id"=\>".monitoring-logstash"} > logstash\_1 | [2019-10-08T17:35:53,651][INFO][logstash.agent] Pipelines running {:count=\>2, :running\_pipelines=\>[:main, :".monitoring-logstash"], :non\_running\_pipelines=\>} > logstash\_1 | Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary. > logstash\_1 | [2019-10-08T17:35:54,104][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=\>9600} > logstash\_1 | [2019-10-08T17:35:55,074][INFO][logstash.inputs.jdbc] (0.053201s) select \* from mytable > logstash\_1 | [2019-10-08T17:35:57,814][INFO][logstash.javapipeline] Pipeline terminated {"pipeline.id"=\>".monitoring-logstash"} > logstash\_1 | [2019-10-08T17:35:58,586][INFO][logstash.runner] Logstash shut down. ---
**Author:** ![happycampo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/happycampo/32/55494_2.png) [@happycampo](https://discuss.elastic.co/u/happycampo)\ **Post date:** [October 9, 2019, 4:32pm UTC](https://discuss.elastic.co/t/pipeline-terminated-pipeline-id-monitoring-logstash-logstash-shutdown/202709/3 "2019-10-09T16:32:26Z")
Anyone? This seems to be happening on a consistent basis. If I start from scratch everything works. Then I shut all the containers down and modify the pipeline/logstash.conf and it breaks. Not sure how to stop the .monitoring-logstash pipeline from killing the container... ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [October 9, 2019, 4:38pm UTC](https://discuss.elastic.co/t/pipeline-terminated-pipeline-id-monitoring-logstash-logstash-shutdown/202709/4 "2019-10-09T16:38:48Z")
You do not have a schedule on your jdbc input, so after it has executed the query once there is nothing left for logstash to do, so it exits. ---
**Author:** ![happycampo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/happycampo/32/55494_2.png) [@happycampo](https://discuss.elastic.co/u/happycampo)\ **Post date:** [October 9, 2019, 5:44pm UTC](https://discuss.elastic.co/t/pipeline-terminated-pipeline-id-monitoring-logstash-logstash-shutdown/202709/5 "2019-10-09T17:44:53Z")
> [@Badger](#): > > You do not have a schedule on your jdbc input, so after it has executed the query once there is nothing left for logstash to do, so it exits. Thanks. I tried that too. It turns out I needed to add the elastic output section I guess becasue of monitoring being setup by default. Now I'm struggling with reading files... ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 6, 2019, 5:45pm UTC](https://discuss.elastic.co/t/pipeline-terminated-pipeline-id-monitoring-logstash-logstash-shutdown/202709/6 "2019-11-06T17:45:03Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.