My pipeline consists of three JDBC inputs, a set of common filters and one output stage to Elasticsearch. The three JDBC inputs fetch similar data from three different tables, the aggregate filter groups the data by a particular field that is present in all three tables, while the output filter dyna…

You'll want to tag each input, then use a agg filter with a conditional for each to make sure they are grouped.

Does this mean it is not possible to run separate inputs sequentially? I'm seeing indexing performance drop by a factor of 10 when the number of inputs increases. I've tried limiting the worker processes, but this seems to have no effect on query execution.

[image] Randy_BoBandy: Does this mean it is not possible to run separate inputs sequentially? Yes.

Discuss the Elastic Stack

How to run queries from multiple JDBC inputs sequentially

Elastic Stack Logstash

warkolm (Mark Walkom) March 21, 2017, 8:18pm 4

Yes.

Reading logs from DB - Avoiding of duplicate records

Topic		Replies	Views
Running multiple SQL Queries in logstash JDBC plugin Logstash	2	683	November 5, 2020
Logstash JDBC input connector Logstash	1	539	June 30, 2017
Do multiple jdbc input run simultaneously or sequentially? Logstash	4	1554	January 17, 2019
Can Logstash jdbc inputs be processed in certain order and how to control that 1 input process has ended? Logstash	6	1455	January 25, 2017
JDBC multiple inputs asynchronous? Logstash	2	273	August 22, 2022

Elasticsearch is a trademark of Elasticsearch BV, registered in the U.S. and in other countries
Trademarks
Terms
Privacy
Brand
Code of Conduct

Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.

How to run queries from multiple JDBC inputs sequentially

Related topics