I want to create an ElasticSearch index with some content of my database. That's a relational database.
What I want to index is document that I have stored in my database in blob fields. I want to index it and perform full text search over them.
I have tryied with a single document using the ingest plugin and it works properly. What I want now is to automitize my process, but I don't know if I can't do it with logstash (with a connection to my database) or what solution to implement. And how about the index for that field?
I'm not sure how LS JDBC input can support blobs TBH. May be something to ask in #logstash group.
What I did in the past was hacking the application which actually stores the data in the database and I was sending attachments from the application to the database to be stored and in the same "transaction" to elasticsearch to be ingested.
Then I wrote a batch which was using somehow the same process to read entirely existing blobs and index them into elasticsearch.
That's the way to go to keep in sync as much as possible both systems IMO.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.