Hi,
I have released JDBC river plugin 2.3.0 for Elasticsearch
https://github.com/jprante/elasticsearch-river-jdbc/releases/tag/2.3.0
Changes:
- new: "column" strategy, thanks to Piotr Śliwa
- new: support for JDBC CallableStatement
- dropped versioning and housekeeper thread, no more deletes
- dropped table strategy, considered as an anti-pattern
- JDBC connections are closed after each river run, avoiding long
outstanding transactions - "index" subsection in river config moved to "jdbc" section for clarity
Source:
Binaries:
https://bintray.com/jprante/elasticsearch-plugins/elasticsearch-river-jdbc/2.3.0/files
Maven site:
http://jprante.github.io/elasticsearch-river-jdbc/
For one of the next releases (3.0.0), I'd like to plan bigger changes. The
river code could be refactored into two parts: a generic framework for
coordinating connectors distributed over many nodes in the cluster (in
contrast to the current autonomous single instance rivers), and a JDBC
connector code. The connectors could get their jobs from a central
dispatcher, fetch data, process it, and push key/value streams back to the
generic framework, which in turn invokes bulk indexing of the generated
documents. Many different fetcher types should be able to use the framework
and run side by side, in parallel, on a single node, or cluster-wide. So
writing a river plugin for such a framework could reduce to writing a
simple connector that just connects to a source and produces key/values for
JSON.
I hope I can somehow leverage the knapsack plugin for a simple
store-and-forward mechanism, so the fetched data of a single river run can
optionally be stored in the filesystem. Each river run assigns a unique ID
to such a data portion. At a scheduled time or by given command, the data
portions can get indexed. The data portion indexing can be replayed in case
of failures.
Jörg
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGnJA5yCbxmj0wr7PK8vjfdMyV_QqJHLkPR%2BYA%2BG4PEKw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.