Ingesting the ElasticSearch index

Elastic Stack Elasticsearch
1

I want to create an ElasticSearch index with some content of my database. That's a relational database.

What I want to index is document that I have stored in my database in blob fields. I want to index it and perform full text search over them.

I have tryied with a single document using the ingest plugin and it works properly. What I want now is to automitize my process, but I don't know if I can't do it with logstash (with a connection to my database) or what solution to implement. And how about the index for that field?

Thanks in advance!

2

Do you mean binary documents like PDF, Open Office... ?
Or JSON documents that you stored as blobs?

3

Binary Documents. Pdf and Word stored as blob.

4

I'm not sure how LS JDBC input can support blobs TBH. May be something to ask in #logstash group.

What I did in the past was hacking the application which actually stores the data in the database and I was sending attachments from the application to the database to be stored and in the same "transaction" to elasticsearch to be ingested.

Then I wrote a batch which was using somehow the same process to read entirely existing blobs and index them into elasticsearch.

That's the way to go to keep in sync as much as possible both systems IMO.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.