ES for primary image storage


(Frank) #1

It has been asked beforehttps://groups.google.com/d/msg/elasticsearch/PrvZMrTktdE/mq0HDZdEqTIJ,
but there was no clear cut answer.

Would ES be suitable for primary storage of images (binary data) as well as
the primary way of retrieving images and presenting to the user?

I my case I consider it for a document-based website, where each document
would have 1-5 images. The obvious way would be to store these images as
certain fields in the corresponding ES-document.

  • Would there be any issues regarding scaling?
  • How would the output be cached?
  • How would this affect memory usage?

/Frank

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ddd7122a-4425-4c81-92d4-f2a909646bd6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Adrien Grand) #2

Hi,

If your images are small (a few kb), you could indeed store them into
Elasticsearch by encoding them using base 64 for example. It would work as
well if they are large, but it could typically hurt search performance by
increasing churn on the filesystem cache. An option could be to have two
Elasticsearch instances (on different hosts), one that store data that is
used for search and stores image metadata and another one that is used as a
key-value store of images. The first instance would be used for search, and
on client side, you could call the 2nd Elasticsearch instance
asynchronously (to not delay the rendering of the page) in order to display
images.

Regarding primary storage on Elasticsearch, we are working on improving
Elasticsearch reliability, but this requires maturity and thorough testing
so we still recommend being able to reindex from another data-source.

On Sat, Apr 12, 2014 at 2:55 PM, Frank frankmannen@gmail.com wrote:

It has been asked beforehttps://groups.google.com/d/msg/elasticsearch/PrvZMrTktdE/mq0HDZdEqTIJ,
but there was no clear cut answer.

Would ES be suitable for primary storage of images (binary data) as well
as the primary way of retrieving images and presenting to the user?

I my case I consider it for a document-based website, where each document
would have 1-5 images. The obvious way would be to store these images as
certain fields in the corresponding ES-document.

  • Would there be any issues regarding scaling?
  • How would the output be cached?
  • How would this affect memory usage?

/Frank

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/ddd7122a-4425-4c81-92d4-f2a909646bd6%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/ddd7122a-4425-4c81-92d4-f2a909646bd6%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Adrien Grand

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j78KHVro%3DDJ7xk-hXDha77T6ycWWZ2Z9jsKmXs8OQypWQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(system) #3