Dynamic query in jdbc plugin

Elastic Stack Logstash
1

I got a mysql(or any other relational database) with a table like this:

id,name,prod
11,alex,car
22,alice,ship
33,bob,airplane

and I have messages going through logstash pipeline like this:

11,20210215,ford,....
22.20210216,renault,...
33,20210217,toyota,...

and I want to use jdbc plugin to perform a query like this:
SELECT name where id=$id;
and grab the name where id is dynamicaly be read from the message and replace it .
after that the messages should be like this:

alex,20210215,ford,....
alice.20210216,renault,...
bob,20210217,toyota,...

I appreciate if anyone can help me with this.

2

Hello Niki,

Have you already checked the example in the documentation - I think this explains your usecase really well: Jdbc_streaming filter plugin | Logstash Reference [7.11] | Elastic

Or do you have a specific problem with the filter you need help with?

Best regards
Wolfram

3

Hello
I read the documentation and I thought it is supposed to be a constant query on each filter but based on my above example my jdbc filter should be something like this:

filter {
jdbc_streaming {
jdbc_driver_library => "/path/to/mysql-connector-java-5.1.34-bin.jar"
jdbc_driver_class => "com.mysql.jdbc.Driver"
jdbc_connection_string => "jdbc:mysql://localhost:3306/mydatabase"
jdbc_user => "me"
jdbc_password => "secret"
statement => "select name from mytable WHERE id= :id"
parameters => { "id" => "id"}
target => "name"
}
}
am i correct?
thank you for the answer.

4

Yes, this is correct

1 Like
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.