Hello,
We are working on setting up a search with document contents (pdf/docx etc.) where permissions comes from a database.
I've gotten all the data into Elastic with the help of Logstash and FSCrawler.
However they are two different indexes and I'm out of ideas how to "merge" them. From someone who comes from doing lots of SQL, I would use JOIN on those indexes to a new view to search on content (from the index created by FSCrawler) with the permissions coming from SQL server Logstash.
The Logstash index has a column "filename" which matches "path.virtual" from FSCrawler.
Thanks for your fast reply! I've tried to look into how to do that as well and read about "denormalization".
What could be a good approach for this with the current setup of Logstash with FSCrawler?
The result is I can't match with the enrich/ingest/pipeline.
This is the difference:
Logstash JDBC plugin "path": "\"\\publicerat\\IN0010.pdf\"",
FSCrawler: "virtual": """\publicerat\IN0010.pdf""",
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.