Storing files into ES database

Elastic Stack Elasticsearch
1

Hello,

I'd like to store files (.csv, .pdf...) into ES database so I can search on their content and visualize relevant informations into a kibana dashboard. Those files are reports of transactions.
After a few searches, I read about mapper-attachment and ElasticWareHouse.

Do anyone know what is the best way to store those files?

Thanks for your help,
Noémie

1 Like
2

Hey Noemie

If you go with the mapper attachments way, please be aware that it's deprecated and you are encouraged using ingest-attachment from 5.0 (GA not available yet).

Storing big BASE64 Content (binary files) in Elasticsearch is not ideal IMO. If you need to store the content somewhere, I'd use something like HDFS or Couchdb. But definitely I'd extract the content and index into Elasticsearch.

That's exactly what I did at my former job.

3

Thank you for this answer.

Is it possible then to make the link between a content and index into Elasticsearch and the file stored in the other db? Like a path or something like thatso the user knows where the file he is looking for is in case this user wants to read the file in addition to the kibana view?

4

Definitely store the file URL in Elasticsearch.
In Kibana there is an option to replace a column content by something else (like a link).

5

ok. Thank you for your answers :slight_smile: