# Multiple logstash intances **URL:** \ **Category:** Logstash\ **Created:** [March 23, 2020, 12:38pm UTC](https://discuss.elastic.co/t/multiple-logstash-intances/224673 "2020-03-23T12:38:37Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![hariskhalique](https://avatars.discourse-cdn.com/v4/letter/h/a3d4f5/32.png) [@hariskhalique](https://discuss.elastic.co/u/hariskhalique)\ **Post date:** [March 23, 2020, 12:38pm UTC](https://discuss.elastic.co/t/multiple-logstash-intances/224673/1 "2020-03-23T12:38:37Z")
Hi, I am trying to run multiple instances of logstash using multiple pipeline. `- pipeline.id: product_info path.config: "/home/ubuntu/dev/V4QaShop/logstash/products.conf" #pipeline.workers: 3 path.data: "/home/ubuntu/dev/V4QaShop/logstash/product" - pipeline.id: product_detail path.config: "/home/ubuntu/dev/V4QaShop/logstash/products_detail.conf" #queue.type: persisted path.data: "/home/ubuntu/dev/V4QaShop/logstash/productDetail"` after running logstash getting following error. > Logstash could not be started because there is already another instance using the configured data directory. If you wish to run multiple instances, you must change the "path.data" setting. no other instance is running can any one help me thanks ---
**Author:** ![rcowart](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rcowart/32/88091_2.png) [@rcowart](https://discuss.elastic.co/u/rcowart)\ **Post date:** [March 25, 2020, 9:14am UTC](https://discuss.elastic.co/t/multiple-logstash-intances/224673/2 "2020-03-25T09:14:16Z")
You should not be setting `path.data` in `pipelines.yml`. The configuration options supported in `pipeline.yml` are: - config.debug - config.support\_escapes - config.reload.automatic - config.reload.interval - config.string - dead\_letter\_queue.enable - dead\_letter\_queue.max\_bytes - metric.collect - pipeline.java\_execution - pipeline.plugin\_classloaders - path.config - path.dead\_letter\_queue - path.queue - pipeline.batch.delay - pipeline.batch.size - pipeline.id - pipeline.reloadable - pipeline.system - pipeline.workers - queue.checkpoint.acks - queue.checkpoint.interval - queue.checkpoint.writes - queue.checkpoint.retry - queue.drain - queue.max\_bytes - queue.max\_events - queue.page\_capacity - queue.type Rob [![GitHub](https://us1.discourse-cdn.com/elastic/original/3X/6/f/6f8ae834f16b1a02d31607317669716807844d84.png)](https://github.com/robcowart) [![YouTube](https://us1.discourse-cdn.com/elastic/original/3X/4/3/43b9b81a8c93786219985aeb5335c1c323449053.png)](https://www.youtube.com/channel/UCivWvTx1DwrWNcDLV58kmOg) [![LinkedIn](https://us1.discourse-cdn.com/elastic/original/3X/6/7/674f3370d0f0542ddc5e408516beb1b7edd6c1bf.png)](https://www.linkedin.com/in/robertcowart/) **[How to install Elasticsearch & Kibana on Ubuntu - incl. hardware recommendations](https://www.youtube.com/watch?v=gZb7HpVOges)** **[What is the best storage technology for Elasticsearch?](https://www.youtube.com/watch?v=nKUpfJCBiS4)** ---
**Author:** ![hariskhalique](https://avatars.discourse-cdn.com/v4/letter/h/a3d4f5/32.png) [@hariskhalique](https://discuss.elastic.co/u/hariskhalique)\ **Post date:** [March 30, 2020, 3:50pm UTC](https://discuss.elastic.co/t/multiple-logstash-intances/224673/3 "2020-03-30T15:50:02Z")
that really help full, even better if you can share what does those option means any link will be help full. Haris ---
**Author:** ![rcowart](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rcowart/32/88091_2.png) [@rcowart](https://discuss.elastic.co/u/rcowart)\ **Post date:** [March 30, 2020, 4:02pm UTC](https://discuss.elastic.co/t/multiple-logstash-intances/224673/4 "2020-03-30T16:02:57Z")
Just a google search away... [https://www.elastic.co/guide/en/logstash/7.6/logstash-settings-file.html](https://www.elastic.co/guide/en/logstash/7.6/logstash-settings-file.html) ---
**Author:** ![hariskhalique](https://avatars.discourse-cdn.com/v4/letter/h/a3d4f5/32.png) [@hariskhalique](https://discuss.elastic.co/u/hariskhalique)\ **Post date:** [March 31, 2020, 7:31pm UTC](https://discuss.elastic.co/t/multiple-logstash-intances/224673/5 "2020-03-31T19:31:04Z")
Following is my config file > input { > jdbc { > jdbc\_driver\_library =\> "/usr/share/java/mysql.jar" > jdbc\_driver\_class =\> "com.mysql.jdbc.Driver" > jdbc\_connection\_string =\> "jdbc:mysql://localhost:3309/v4qadb" > jdbc\_user =\> "v4qashop" > jdbc\_password =\> "V4admin09&#!" > jdbc\_paging\_enabled =\> true > jdbc\_page\_size =\> 10000 > tracking\_column =\> "pro\_updated\_date" > use\_column\_value=\>true > # statement =\> "select \* from `es_product_detail_info` where `pro_updated_date` \>=:sql\_last\_value" > schedule =\> "\*/10101010101010101010 \* \* \* \*" > statement =\> "SELECT > `item_id`,`product_id`,`item_color_id`,`item_size_id`,`item_sku`,`item_image`,`item_price`,`brand_id`,`control_number`,`description`, > `display_product_color_id`,`item_details`,`item_name`,`gender`,`pro_updated_date` ,`shopify_product_id`,`image_product_name`,`image_sort_order`, > `image_title`,`image_id`,`updated_at`,`document_id`,`item_color_image`,`item_v4_size_title`,`cat_name`,`cat_shopify_collection_id` > from es\_product\_info where `pro_updated_date` \>=:sql\_last\_value" > > } > } > output { > elasticsearch { > document\_id=\> "%{document\_id}" > document\_type =\> "doc" > index =\> "product\_detail\_info" > hosts =\> ["[http://localhost:9200](http://localhost:9200)"] > } > stdout{ > codec =\> rubydebug > } > } Is there is a limit on free version of ELK total records are 547242 it stops at 270034. Plus I am running on multiple pipelines yaml file is > - pipeline.id: morph\_styles > path.config: "/home/ubuntu/dev/V4QaShop/logstash/morph\_styles.conf" > pipeline.workers: 6 > #batch: > # size: 50000 > > # delay: ${BATCH\_DELAY:50} > - pipeline.id: demo\_morph\_styles > path.config: "/home/ubuntu/dev/V4QaShop/logstash/demo\_morph\_styles.conf" > pipeline.workers: 6 > #batch: > # size: 50000 > > # delay: ${BATCH\_DELAY:50} > - pipeline.id: product\_detail\_info > #batch: > # size: 20000 > > # delay: ${BATCH\_DELAY:50} > path.config: "/home/ubuntu/dev/V4QaShop/logstash/products\_detail.conf" > queue.type: persisted > #pipline.workers: 5 other never get a chance to run why ? Haris ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 28, 2020, 7:31pm UTC](https://discuss.elastic.co/t/multiple-logstash-intances/224673/6 "2020-04-28T19:31:14Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.