# In logstash can we have multiple inputs and combine the result set and process the combined data to one output?

**URL:** <https://discuss.elastic.co/t/in-logstash-can-we-have-multiple-inputs-and-combine-the-result-set-and-process-the-combined-data-to-one-output/129281>\
**Category:** Logstash\
**Created:** [April 24, 2018, 10:12am UTC](https://discuss.elastic.co/t/in-logstash-can-we-have-multiple-inputs-and-combine-the-result-set-and-process-the-combined-data-to-one-output/129281 "2018-04-24T10:12:05Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![manjur](https://avatars.discourse-cdn.com/v4/letter/m/5f8ce5/32.png) [@manjur](https://discuss.elastic.co/u/manjur)\
**Post date:** [April 24, 2018, 10:12am UTC](https://discuss.elastic.co/t/in-logstash-can-we-have-multiple-inputs-and-combine-the-result-set-and-process-the-combined-data-to-one-output/129281/1 "2018-04-24T10:12:05Z")

</div>

Hi ,  
I am using a config file like below. In the input section I am pulling the data from different database. I want to perform join and combine results in filter section. basically I just want to merge the result set ,after that the combined data should go to one output .  
input  
{  
jdbc  
{  
//select \* from db1.table1

}

jdbc  
{  
//select \* from db2.table1  
}

}

filter  
{  
}

output  
{  
// single output.  
}

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [April 24, 2018, 10:29am UTC](https://discuss.elastic.co/t/in-logstash-can-we-have-multiple-inputs-and-combine-the-result-set-and-process-the-combined-data-to-one-output/129281/2 "2018-04-24T10:29:19Z")

</div>

Logstash handles streams of events, so I do not think you can combine the data from the two inputs. You may however be able to use the [jdbc streaming](https://www.elastic.co/guide/en/logstash/current/plugins-filters-jdbc_streaming.html) or [jdbc static](https://www.elastic.co/guide/en/logstash/current/plugins-filters-jdbc_static.html) plugins to enrich the data generated through one of the inputs with data from the other database table.

---

<div class="post-metadata">

**Author:** ![manjur](https://avatars.discourse-cdn.com/v4/letter/m/5f8ce5/32.png) [@manjur](https://discuss.elastic.co/u/manjur)\
**Post date:** [April 24, 2018, 12:48pm UTC](https://discuss.elastic.co/t/in-logstash-can-we-have-multiple-inputs-and-combine-the-result-set-and-process-the-combined-data-to-one-output/129281/3 "2018-04-24T12:48:24Z")

</div>

as I can see we can give one jdbc\_connection\_string and deals with multiple tables. Is there any way we can give multiple connection string?

---

<div class="post-metadata">

**Author:** ![manjur](https://avatars.discourse-cdn.com/v4/letter/m/5f8ce5/32.png) [@manjur](https://discuss.elastic.co/u/manjur)\
**Post date:** [April 24, 2018, 12:55pm UTC](https://discuss.elastic.co/t/in-logstash-can-we-have-multiple-inputs-and-combine-the-result-set-and-process-the-combined-data-to-one-output/129281/4 "2018-04-24T12:55:47Z")

</div>

can we give jdbc\_connection\_string in the loaders section?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [April 24, 2018, 12:57pm UTC](https://discuss.elastic.co/t/in-logstash-can-we-have-multiple-inputs-and-combine-the-result-set-and-process-the-combined-data-to-one-output/129281/5 "2018-04-24T12:57:51Z")

</div>

Can you explain what the data represents, where it resides and how you are looking to merge them?

---

<div class="post-metadata">

**Author:** ![manjur](https://avatars.discourse-cdn.com/v4/letter/m/5f8ce5/32.png) [@manjur](https://discuss.elastic.co/u/manjur)\
**Post date:** [April 24, 2018, 1:04pm UTC](https://discuss.elastic.co/t/in-logstash-can-we-have-multiple-inputs-and-combine-the-result-set-and-process-the-combined-data-to-one-output/129281/6 "2018-04-24T13:04:37Z")

</div>

lets say table1 from database1 has 100 rows having columns id and name. Similarly table2 from database2 has 100 rows having columns id, and city. I just want to perform join on both the tables and apply some filters on the result set, I can do this in sql file but I just want to know how logstash will behave to this scenario.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [April 24, 2018, 1:06pm UTC](https://discuss.elastic.co/t/in-logstash-can-we-have-multiple-inputs-and-combine-the-result-set-and-process-the-combined-data-to-one-output/129281/7 "2018-04-24T13:06:13Z")

</div>

If you can, I would recommend performing the join in the database.

---

<div class="post-metadata">

**Author:** ![manjur](https://avatars.discourse-cdn.com/v4/letter/m/5f8ce5/32.png) [@manjur](https://discuss.elastic.co/u/manjur)\
**Post date:** [April 24, 2018, 1:09pm UTC](https://discuss.elastic.co/t/in-logstash-can-we-have-multiple-inputs-and-combine-the-result-set-and-process-the-combined-data-to-one-output/129281/8 "2018-04-24T13:09:14Z")

</div>

sure. But do we have connection string option in the below section of loader of jdbc static plug in.

loaders =\> [  
{  
id =\> "remote-servers"  
query =\> "select ip, descr from ref.local\_ips order by ip"  
local\_table =\> "servers"  
},

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [April 24, 2018, 1:10pm UTC](https://discuss.elastic.co/t/in-logstash-can-we-have-multiple-inputs-and-combine-the-result-set-and-process-the-combined-data-to-one-output/129281/9 "2018-04-24T13:10:43Z")

</div>

If you are performing the join in the database prior to the data being read into Logstash through the input plugin, you will not need the plugins I linked to.

---

<div class="post-metadata">

**Author:** ![manjur](https://avatars.discourse-cdn.com/v4/letter/m/5f8ce5/32.png) [@manjur](https://discuss.elastic.co/u/manjur)\
**Post date:** [April 24, 2018, 1:11pm UTC](https://discuss.elastic.co/t/in-logstash-can-we-have-multiple-inputs-and-combine-the-result-set-and-process-the-combined-data-to-one-output/129281/10 "2018-04-24T13:11:40Z")

</div>

okay. thank you

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 22, 2018, 1:11pm UTC](https://discuss.elastic.co/t/in-logstash-can-we-have-multiple-inputs-and-combine-the-result-set-and-process-the-combined-data-to-one-output/129281/11 "2018-05-22T13:11:43Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
