Can we have multiple input and output section in logstash config file

Elastic Stack Beats
1

can we have multiple input and output section in logstash config file. So that we can deal with multiple indexes in same run.

2

Hi, can you be more explicit on what you want to achieve? Which is your beats setup and how do you want it to look like in Elasticsearch?

3

my logstash.config file looks like. i am transferring data from sqlserver to ES .

input {
jdbc {
jdbc_driver_library => "C:\Users\manjur.gani\Desktop\ElasticSearch\logstash-6.2.3\bin\mysql-connector-java-5.1.46-bin.jar"
jdbc_driver_class => "com.mysql.jdbc.Driver"
jdbc_connection_string => "jdbc:mysql://localhost:3306/sys"
jdbc_user => "root"
jdbc_password => "ganiclub"
statement => "SELECT * from sys_employee"
schedule => "* * * * *"
}
}
filter{
}
output
{
elasticsearch {
hosts => ["localhost:9200"]
index => "employee1"
}

=>what i want to achieve , i should able to give multiple jdbc connection and in output i need multiple indexes like

output
{
elasticsearch {
hosts => ["localhost:9200"]
index => "employee1"
Index=>""emp2"
index=>"emp3"
}

4

This can be achieved fairly easy.

You can have multiple inputs with the same "type" e.g. jdbc.
just add those multiple times, is there a difference in the username you are using, or only the statement?

In the output box, you could do something like if "root" in jdbc_user { output 1 }
if the username is always the same I would go for a tagging, this would add an extra field to each of your line but, it will help you with the output. For each input you would need to add tags => "jdbc-1afterwards you can do the same thing in the output with if "jdbc-1" in [tags]

5

hi philip,

i did it in the same way , but i don't want to give jdbc connection string each time, i have one jdbc connection and in the statement part ex:"select * from emp", i need multiple statement as input for multiple indexes. Is there any way where i can do like below

input
{
jdbc {
jdbc_driver_library => "C:\Users\manjur.gani\Desktop\ElasticSearch\logstash-6.2.3\bin\mysql-connector-java-5.1.46-bin.jar"
jdbc_driver_class => "com.mysql.jdbc.Driver"
jdbc_connection_string => "jdbc:mysql://localhost:3306/sys"
jdbc_user => "root"
jdbc_password => "ganiclub"
}

{
statement=>select * from emp
type=>"emp"
}

{
statement=>select * from manager
type=>"mgr"
}

}

6

https://stackoverflow.com/questions/37613611/multiple-inputs-on-logstash-jdbc no it is not possible to do anything like that. When you open up a jdbc it expects all the fields like jdbc_driver_library to be filled. Please correct me if I am wrong.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.