# How to run multiple instances on one machine using the jdbc input plugin

**URL:** https://discuss.elastic.co/t/how-to-run-multiple-instances-on-one-machine-using-the-jdbc-input-plugin/97704
**Category:** Logstash
**Created:** [August 21, 2017, 3:46am UTC](https://discuss.elastic.co/t/how-to-run-multiple-instances-on-one-machine-using-the-jdbc-input-plugin/97704 "2017-08-21T03:46:35Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![JCereal](https://avatars.discourse-cdn.com/v4/letter/j/3da27b/32.png) [@JCereal](https://discuss.elastic.co/u/JCereal)
#### Post date: [August 21, 2017, 3:46am UTC](https://discuss.elastic.co/t/how-to-run-multiple-instances-on-one-machine-using-the-jdbc-input-plugin/97704/1 "2017-08-21T03:46:35Z")

</div>

Hi all,

I have got 6 configuration files. All of them are using jbdc plugin to read data from MySQL every minute as input and elastic search as output. Each instance write into different index, so I have got six config files and six indexes.

While I run the 6 instances one by one, they all work fine. But if I put those 6 config files into a directory, and start the logstash manually using the ‘-f’ parameter pointing to the directory, the documents are not ingested into different index properly. All the documents just messed up.

I'm using CentOS 7.2, Elastic Stack 5.5.1. And I install the logstash and elastic search by simply unzipping the tar file.

Is there any solution to this problem? I really need to monitor the changes in these 6 MySQL tables simultaneously.

Here are two of my config files:

1:

```
input {
  jdbc {
		jdbc_connection_string => "jdbc:mysql:// *****"
		jdbc_user => " *****"
		jdbc_password => " *****"
		
		jdbc_driver_library => "/opt/elasticstack/logstash-5.5.1/lib/mysql-connector-java-5.1.42-bin.jar"
		jdbc_driver_class => "com.mysql.jdbc.Driver"

		statement_filepath => "/home/elsearch/data_migration_v0.2/sql/thread_mysql2es.sql"
		
		schedule => "* * * * *"
		last_run_metadata_path => "/home/elsearch/.logstash_jdbc_last_run_thread"

		type => "thread"
  }
}

output {
  elasticsearch {
  	hosts => ["https:// **** :9200"]
  	index => "thread"
   	document_id => "%{id}"

	user => logstash_lsj
	password => logstash_lsj123
	ssl => true
	cacert => '/opt/elasticstack/logstash-5.5.1/ca.pem'
  }
}

```

2:

```
input {
  jdbc {
		jdbc_connection_string => "jdbc:mysql:// *****"
		jdbc_user => " *****"
		jdbc_password => " *****"
		
		jdbc_driver_library => "/opt/elasticstack/logstash-5.5.1/lib/mysql-connector-java-5.1.42-bin.jar"
		jdbc_driver_class => "com.mysql.jdbc.Driver"

		statement_filepath => "/home/elsearch/data_migration_v0.2/sql/topic_mysql2es.sql"
		
		schedule => "* * * * *"
		last_run_metadata_path => "/home/elsearch/.logstash_jdbc_last_run_topic"

		type => "topic"
  }
}

output {
  elasticsearch {
  	hosts => ["https:// **** :9200"]
  	index => "topic"
   	document_id => "%{id}"

	user => logstash_lsj
	password => logstash_lsj123
	ssl => true
	cacert => '/opt/elasticstack/logstash-5.5.1/ca.pem'
  }
}

```

So much thanks!

---

<div class="post-metadata">

### Author: ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)
#### Post date: [August 21, 2017, 1:32pm UTC](https://discuss.elastic.co/t/how-to-run-multiple-instances-on-one-machine-using-the-jdbc-input-plugin/97704/2 "2017-08-21T13:32:53Z")

</div>

Until Logstash 6.0 is officially released, you will have issues with trying to load all of those pipelines in a single Logstash instance. The reason is that until then, all configurations in that directory are merged into a single pipeline. Technically, after then, they will be as well, but you can identify individual pipelines in the new [pipelines.yml](https://www.elastic.co/blog/logstash-multiple-pipelines) file.

That blog post I just linked explains the problem, the ways users have worked around it in the past, and the new way coming in Logstash 6.0.

---

<div class="post-metadata">

### Author: ![JCereal](https://avatars.discourse-cdn.com/v4/letter/j/3da27b/32.png) [@JCereal](https://discuss.elastic.co/u/JCereal)
#### Post date: [August 22, 2017, 2:21am UTC](https://discuss.elastic.co/t/how-to-run-multiple-instances-on-one-machine-using-the-jdbc-input-plugin/97704/3 "2017-08-22T02:21:05Z")

</div>

Thank you so much, the blog helped a lot. I will try out the "tag" way and multi instances way. I will also play with the 6.0 beta om my own laptop.

Looking forward to the general release of the 6.0 version. 😀

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [September 19, 2017, 2:21am UTC](https://discuss.elastic.co/t/how-to-run-multiple-instances-on-one-machine-using-the-jdbc-input-plugin/97704/4 "2017-09-19T02:21:13Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
