# Multiple logstash instances for ecommerce

**URL:** <https://discuss.elastic.co/t/multiple-logstash-instances-for-ecommerce/108515>\
**Category:** Logstash\
**Created:** [November 21, 2017, 6:11am UTC](https://discuss.elastic.co/t/multiple-logstash-instances-for-ecommerce/108515 "2017-11-21T06:11:12Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Gobinda\_Nandi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gobinda_nandi/32/24506_2.png) [@Gobinda\_Nandi](https://discuss.elastic.co/u/Gobinda_Nandi)\
**Post date:** [November 21, 2017, 6:11am UTC](https://discuss.elastic.co/t/multiple-logstash-instances-for-ecommerce/108515/1 "2017-11-21T06:11:12Z")

</div>

hi,  
i want to implement ELK for an ecommerce. Project is ready on php and mysql. I want to search product using EL.  
so i have to migrate mysql data to logstash index right. But there are lots of other conditions and condition based query is running.  
What i need to know , for this kinda situation do i need to use multiple logstash instances ? if so what is conf file template for multiple logstash instances.  
Or there are other easy way ??

Thanks .

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [November 21, 2017, 6:29am UTC](https://discuss.elastic.co/t/multiple-logstash-instances-for-ecommerce/108515/2 "2017-11-21T06:29:37Z")

</div>

There's nothing in the description you've given that indicates a need to run multiple Logstash instances (especially not with the multi-pipeline support in Logstash 6.0), but as you're not giving any details it's hard to tell.

---

<div class="post-metadata">

**Author:** ![Gobinda\_Nandi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gobinda_nandi/32/24506_2.png) [@Gobinda\_Nandi](https://discuss.elastic.co/u/Gobinda_Nandi)\
**Post date:** [November 21, 2017, 8:05am UTC](https://discuss.elastic.co/t/multiple-logstash-instances-for-ecommerce/108515/3 "2017-11-21T08:05:30Z")

</div>

I have written this code (.conf file ) for multiple statement . please tell me if i am wrong.

```
input {

	#first table

	jdbc {
		jdbc_driver_library => "/mysql-connector-java-5.1.44-bin.jar"
		jdbc_driver_class => "com.mysql.jdbc.Driver"
		jdbc_connection_string => "jdbc:mysql://localhost:3306/database_name"
		jdbc_user => "root"
		jdbc_password => ""
		schedule => "* * * * *"
		statement => "SELECT @slno:=@slno+1 aut_es_1, es_qry_tbl.* FROM (SELECT * FROM `my_test_121223212`) es_qry_tbl, (SELECT @slno:=0) es_tbl"
    	type => "user"
		add_field => { "SQLDescription" => "SQLDesc1" }
		use_column_value => true
    	tracking_column => "aut_es_1"
	}

	#second table

	jdbc {
		jdbc_driver_library => "/mysql-connector-java-5.1.44-bin.jar"
		jdbc_driver_class => "com.mysql.jdbc.Driver"
		jdbc_connection_string => "jdbc:mysql://localhost:3306/database_name"
		jdbc_user => "root"
		jdbc_password => ""
		schedule => "* * * * *"
		statement => "SELECT @slno:=@slno+1 aut_es_2, es_qry_tbl.* FROM (SELECT * FROM `my_test1212`) es_qry_tbl, (SELECT @slno:=0) es_tbl"
    	type => "user"
		add_field => { "SQLDescription" => "SQLDesc2" }
		use_column_value => true
    	tracking_column => "aut_es_2"
	}

	#third table

	jdbc {
		jdbc_driver_library => "/mysql-connector-java-5.1.44-bin.jar"
		jdbc_driver_class => "com.mysql.jdbc.Driver"
		jdbc_connection_string => "jdbc:mysql://localhost:3306/database_name"
		jdbc_user => "root"
		jdbc_password => ""
		schedule => "* * * * *"
		statement => "SELECT @slno:=@slno+1 aut_es_3, es_qry_tbl.* FROM (SELECT * FROM `my_test`) es_qry_tbl, (SELECT @slno:=0) es_tbl"
    	type => "user"
		add_field => { "SQLDescription" => "SQLDesc3" }
		use_column_value => true
    	tracking_column => "aut_es_3"
	}
}

filter {
  mutate {
    add_field => {
      "[@metadata][document_id]" => "%{aut_es_1}%{aut_es_2}%{aut_es_3}"
    }
  }
}

output {
	#stdout {codec => rubydebug}
	elasticsearch {
		hosts => "localhost:9200"
		index => "seacrh_gobinda_ind"
		document_id => "%{[@metadata][document_id]}"
	}
}
```

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [November 21, 2017, 8:17am UTC](https://discuss.elastic.co/t/multiple-logstash-instances-for-ecommerce/108515/4 "2017-11-21T08:17:12Z")

</div>

> ```
> "[@metadata][document_id]" => "%{aut_es_1}%{aut_es_2}%{aut_es_3}"
> 
> ```

This won't work since `aut_es_X` are fields in different events. Logstash won't join the events, but can't you fetch everything in a single SQL query?

---

<div class="post-metadata">

**Author:** ![Gobinda\_Nandi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gobinda_nandi/32/24506_2.png) [@Gobinda\_Nandi](https://discuss.elastic.co/u/Gobinda_Nandi)\
**Post date:** [November 21, 2017, 8:31am UTC](https://discuss.elastic.co/t/multiple-logstash-instances-for-ecommerce/108515/5 "2017-11-21T08:31:27Z")

</div>

no i can't , these are all dependent/conditional queries.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 19, 2017, 8:31am UTC](https://discuss.elastic.co/t/multiple-logstash-instances-for-ecommerce/108515/6 "2017-12-19T08:31:33Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.