# While using pipeline Error: Address already in use Exception **URL:** \ **Category:** Logstash\ **Created:** [March 21, 2018, 4:55pm UTC](https://discuss.elastic.co/t/while-using-pipeline-error-address-already-in-use-exception/125039 "2018-03-21T16:55:50Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![chandukreddi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chandukreddi/32/29241_2.png) [@chandukreddi](https://discuss.elastic.co/u/chandukreddi)\ **Post date:** [March 21, 2018, 4:55pm UTC](https://discuss.elastic.co/t/while-using-pipeline-error-address-already-in-use-exception/125039/1 "2018-03-21T16:55:50Z")
Hello Experts, when I was trying to run logstash using pipeline.yml I am running into below error. I don't find any logstash instance running **[root@localhost config]# ps -ef | grep logstash** **root 26227 22729 0 09:11 pts/1 00:00:00 grep --color=auto logstash** ---
**Author:** ![chandukreddi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chandukreddi/32/29241_2.png) [@chandukreddi](https://discuss.elastic.co/u/chandukreddi)\ **Post date:** [March 21, 2018, 4:56pm UTC](https://discuss.elastic.co/t/while-using-pipeline-error-address-already-in-use-exception/125039/2 "2018-03-21T16:56:22Z")
**Error:** [root@localhost config]# /opt/logstash-6.1.3/bin/logstash 2018-03-21 08:54:47,125 main ERROR Unable to locate appender "${sys:ls.log.format}\_console" for logger config "root" 2018-03-21 08:54:47,126 main ERROR Unable to locate appender "${sys:ls.log.format}\_rolling" for logger config "root" 2018-03-21 08:54:47,126 main ERROR Unable to locate appender "${sys:ls.log.format}\_rolling\_slowlog" for logger config "slowlog" 2018-03-21 08:54:47,127 main ERROR Unable to locate appender "${sys:ls.log.format}\_console\_slowlog" for logger config "slowlog" 2018-03-21 08:54:48,957 main ERROR Unable to locate appender "${sys:ls.log.format}\_console" for logger config "root" 2018-03-21 08:54:48,958 main ERROR Unable to locate appender "${sys:ls.log.format}\_rolling" for logger config "root" 2018-03-21 08:54:48,958 main ERROR Unable to locate appender "${sys:ls.log.format}\_rolling\_slowlog" for logger config "slowlog" 2018-03-21 08:54:48,959 main ERROR Unable to locate appender "${sys:ls.log.format}\_console\_slowlog" for logger config "slowlog" Sending Logstash's logs to /opt/data/logs which is now configured via log4j2.properties [2018-03-21T08:54:49,173][INFO][logstash.modules.scaffold] Initializing module {:module\_name=\>"fb\_apache", :directory=\>"/opt/logstash-6.1.3/modules/fb\_apache/configuration"} [2018-03-21T08:54:49,184][INFO][logstash.modules.scaffold] Initializing module {:module\_name=\>"netflow", :directory=\>"/opt/logstash-6.1.3/modules/netflow/configuration"} [2018-03-21T08:54:50,035][INFO][logstash.runner] Starting Logstash {"logstash.version"=\>"6.1.3"} [2018-03-21T08:54:50,744][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=\>9600} [2018-03-21T08:54:56,804][INFO][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=\>{:removed=\>[], :added=\>[[http://10.1.27.6:9200/](http://10.1.27.6:9200/)]}} [2018-03-21T08:54:56,814][INFO][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck\_url=\>[http://10.1.27.6:9200/](http://10.1.27.6:9200/), :path=\>"/"} [2018-03-21T08:54:57,044][WARN][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=\>"[http://10.1.27.6:9200/](http://10.1.27.6:9200/)"} [2018-03-21T08:54:57,124][INFO][logstash.outputs.elasticsearch] ES Output version determined {:es\_version=\>nil} [2018-03-21T08:54:57,128][WARN][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document \_type {:es\_version=\>6} [2018-03-21T08:54:57,149][INFO][logstash.outputs.elasticsearch] Using mapping template from {:path=\>"cass\_log\_sizing\_2.json"} [2018-03-21T08:54:57,165][INFO][logstash.outputs.elasticsearch] Attempting to install template {:manage\_template=\>{"template"=\>"cassandra\_log", "index\_patterns"=\>["prd-log-_"], "settings"=\>{"index.refresh\_interval"=\>"5s", "index.codec"=\>"best\_compression", "number\_of\_shards"=\>5, "number\_of\_replicas"=\>0}, "aliases"=\>{"logs\_write\_cas"=\>{}}}} [2018-03-21T08:54:57,220][INFO][logstash.outputs.elasticsearch] Installing elasticsearch template to \_template/cassandra\_log [2018-03-21T08:54:57,289][INFO][logstash.outputs.elasticsearch] New Elasticsearch output {:class=\>"LogStash::Outputs::ElasticSearch", :hosts=\>["[//10.1.27.6:9200](https://10.1.27.6:9200)"]} [2018-03-21T08:54:57,851][INFO][logstash.pipeline] Starting pipeline {:pipeline\_id=\>"cassandra-log", "pipeline.workers"=\>2, "pipeline.batch.size"=\>125, "pipeline.batch.delay"=\>5, "pipeline.max\_inflight"=\>250, :thread=\>"#\"} [2018-03-21T08:54:58,170][INFO][logstash.inputs.beats] Beats inputs: Starting input listener {:address=\>"0.0.0.0:5044"} [2018-03-21T08:54:58,224][INFO][logstash.pipeline] Pipeline started {"pipeline.id"=\>"cassandra-log"} [2018-03-21T08:54:58,341][INFO][org.logstash.beats.Server] Starting server on port: 5044 [2018-03-21T08:54:59,286][INFO][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=\>{:removed=\>[], :added=\>[[http://10.1.27.6:9200/](http://10.1.27.6:9200/)]}} [2018-03-21T08:54:59,289][INFO][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck\_url=\>[http://10.1.27.6:9200/](http://10.1.27.6:9200/), :path=\>"/"} [2018-03-21T08:54:59,298][WARN][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=\>"[http://10.1.27.6:9200/](http://10.1.27.6:9200/)"} [2018-03-21T08:54:59,304][INFO][logstash.outputs.elasticsearch] ES Output version determined {:es\_version=\>nil} [2018-03-21T08:54:59,305][WARN][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document \_type {:es\_version=\>6} [2018-03-21T08:54:59,308][INFO][logstash.outputs.elasticsearch] Using mapping template from {:path=\>"swift\_proxy\_log\_sizing\_2.json"} [2018-03-21T08:54:59,310][INFO][logstash.outputs.elasticsearch] Attempting to install template {:manage\_template=\>{"template"=\>"swift\_proxy\_log", "index\_patterns"=\>["swift-proxy-log-_"], "settings"=\>{"index.refresh\_interval"=\>"5s", "index.codec"=\>"best\_compression", "number\_of\_shards"=\>5, "number\_of\_replicas"=\>0}, "aliases"=\>{"logs\_write\_swift"=\>{}}}} [2018-03-21T08:54:59,316][INFO][logstash.outputs.elasticsearch] Installing elasticsearch template to \_template/swift\_proxy\_log [2018-03-21T08:54:59,329][INFO][logstash.outputs.elasticsearch] New Elasticsearch output {:class=\>"LogStash::Outputs::ElasticSearch", :hosts=\>["[//10.1.27.6:9200](https://10.1.27.6:9200)"]} [2018-03-21T08:54:59,414][INFO][logstash.pipeline] Starting pipeline {:pipeline\_id=\>"swiftproxy-log", "pipeline.workers"=\>3, "pipeline.batch.size"=\>125, "pipeline.batch.delay"=\>5, "pipeline.max\_inflight"=\>375, :thread=\>"#\"} [2018-03-21T08:54:59,421][INFO][logstash.inputs.beats] Beats inputs: Starting input listener {:address=\>"0.0.0.0:5044"} [2018-03-21T08:54:59,425][INFO][logstash.pipeline] Pipeline started {"pipeline.id"=\>"swiftproxy-log"} [2018-03-21T08:54:59,434][INFO][org.logstash.beats.Server] Starting server on port: 5044 [2018-03-21T08:54:59,545][INFO][logstash.agent] Pipelines running {:count=\>2, :pipelines=\>["cassandra-log", "swiftproxy-log"]} [2018-03-21T08:55:05,782][ERROR][logstash.pipeline] A plugin had an unrecoverable error. Will restart this plugin. Pipeline\_id:swiftproxy-log Plugin: \5044, id=\>"1643e1f8253a58f0682eaed58daf10e51214458b68e6a05499533e259ad5cc5f", enable\_metric=\>true, codec=\>\"plain\_64879677-b4aa-4b54-87ad-87dd78716278", enable\_metric=\>true, charset=\>"UTF-8"\>, host=\>"0.0.0.0", ssl=\>false, ssl\_verify\_mode=\>"none", ---
**Author:** ![chandukreddi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chandukreddi/32/29241_2.png) [@chandukreddi](https://discuss.elastic.co/u/chandukreddi)\ **Post date:** [March 21, 2018, 4:57pm UTC](https://discuss.elastic.co/t/while-using-pipeline-error-address-already-in-use-exception/125039/3 "2018-03-21T16:57:02Z")
your help appreciated! ---
**Author:** ![chandukreddi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chandukreddi/32/29241_2.png) [@chandukreddi](https://discuss.elastic.co/u/chandukreddi)\ **Post date:** [March 21, 2018, 5:04pm UTC](https://discuss.elastic.co/t/while-using-pipeline-error-address-already-in-use-exception/125039/4 "2018-03-21T17:04:54Z")
Sorry I forgot to add my pipeline.yml config details [root@localhost config]# cat pipelines.yml | head -n 50 ### Cassandra Log - pipeline.id: cassandra-log pipeline.workers: 2 pipeline.batch.size: 125 path.config: "/opt/logstash-6.1.3/config/cassandra-new.conf" ### Swiftproxy log - pipeline.id: swiftproxy-log pipeline.workers: 3 pipeline.batch.size: 125 path.config: "/opt/logstash-6.1.3/config/swift\_proxy.conf" Thanks Chandra ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [March 21, 2018, 7:49pm UTC](https://discuss.elastic.co/t/while-using-pipeline-error-address-already-in-use-exception/125039/5 "2018-03-21T19:49:35Z")
Your pipelines have the same beats port input, you need to change one of them, you can't have two inputs using the same port. ```auto [2018-03-21T08:54:58,224][INFO][logstash.pipeline] Pipeline started {"pipeline.id"=>"cassandra-log"} [2018-03-21T08:54:58,341][INFO][org.logstash.beats.Server] Starting server on port: 5044 ... [2018-03-21T08:54:59,425][INFO][logstash.pipeline] Pipeline started {"pipeline.id"=>"swiftproxy-log"} [2018-03-21T08:54:59,434][INFO][org.logstash.beats.Server] Starting server on port: 5044 ``` ---
**Author:** ![chandukreddi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chandukreddi/32/29241_2.png) [@chandukreddi](https://discuss.elastic.co/u/chandukreddi)\ **Post date:** [March 21, 2018, 8:16pm UTC](https://discuss.elastic.co/t/while-using-pipeline-error-address-already-in-use-exception/125039/6 "2018-03-21T20:16:06Z")
Hi @leandrojmp, How do I change input port? I have 2 logs files on same source host but I have to use 2 different conf files to parse them. Thanks Chandra ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [March 25, 2018, 6:42pm UTC](https://discuss.elastic.co/t/while-using-pipeline-error-address-already-in-use-exception/125039/7 "2018-03-25T18:42:18Z")
> How do I change input port? Change the port number in the beats input as well as the port on the sending side (i.e. one of the Filebeat instances). > I have 2 logs files on same source host but I have to use 2 different conf files to parse them. In your Filebeat configuration, set a field or tag to indicate the kind of log for each prospector and filename pattern. Then you can inspect that field or tag in the Logstash configuration to choose between different set of filters. Then you don't need two pipelines. ---
**Author:** ![chandukreddi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chandukreddi/32/29241_2.png) [@chandukreddi](https://discuss.elastic.co/u/chandukreddi)\ **Post date:** [March 26, 2018, 5:54pm UTC](https://discuss.elastic.co/t/while-using-pipeline-error-address-already-in-use-exception/125039/8 "2018-03-26T17:54:30Z")
Thanks @magnusbaeck.. I appreciate your time ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 23, 2018, 5:54pm UTC](https://discuss.elastic.co/t/while-using-pipeline-error-address-already-in-use-exception/125039/9 "2018-04-23T17:54:36Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.