Can I pass data between 2 input plugin of logstash

Elastic Stack Logstash
1

input {

kafka {

// Pass a value from kafka to jdbc input plugin
}

jdbc {

}

}

2

No, that is not possible. What is it you are looking to achieve?

3

I want fetch a message (and index value (primary_key of a table )) from Kafka Queue and pass this jdbc input so that i can query it and send the data to Elasticsearch

4

How many records/documents would each query return? If you are simply looking up a value based on the key coming from Kafka, you may be able to use the jdbc_streaming plugin for this.

5

Only One key at a time,
You mean I can use Kafka in the Input plugin and then use the Jdbc stream plugin to take input value from kafka and then run a query in the filter section and then in ouput section use elasticsearch right ?

input{
kakfa {}
}

filter {
jdbcstreaming
}

output {
elasticsearch {}
}

Is this right ? Can you share example if you have any ?

6

Something like that. Do not have any example though.

7

But this is possible if implemented ?

8

@Christian_Dahlqvist, is it not possible to configure a TCP output for the first 'stage' and then a TCP input for the second 'stage'?

9

@patrick007 I think it should be possible, but I do not know anything about your data or queries.

@wwalker I am not sure I understand what you are suggesting. Could you please add some more details on why TCP inputs would be useful here?

11

@Christian_Dahlqvist
Thanks for you input :+1:
Used a wrong emoticon first :sweat_smile::sweat_smile:

12

Input from Kafka topic, enrich, output to another Kafka topic.... rinse lather repeat

13

@sm00thindian Did not follow, Can you explain more please

14

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.