I hope I'm in the right category. If not please move this topic to the right one.
I'm already writing an indexing mechanism for pictures to index about 10.000.000 pictures of a friend of mine who's a professional photographer. When this is done he'll also get a new system for storage because the existing one is not able to keep up with the amount of data anymore.
We're planning to use a SAS HBA with spinning disks for the pictures and a bunch of SSDs for cache and Elastic Search. Memory will be between 64 and 128GB. I guess memory can only be replaced by more memory
The only thing I'm not sure about is the processor. As he wants to get a small chassis for the server itself (disks are stored in another chassis), we looked at Supermicro's embedded servers but they all come with the Xeon D.
AFAIK those processors are "better" Intel Atom chips? They usually come with 8 cores, but I'm not sure if they're the "number crunchers" I need.
Has anyone ever run ES on one of those processors?
Maybe has some information about the benchmark?
Any recommendations on the processor to use?
What is better more cores or higher speed per core? (I'd rather go for more cores than higher speeds.)
Another problem I got right now is that I want to store the preview image as base64 blob in ES. The size is around 6-10kb per image after being converted to base64.
As it is usually a bad idea to store blobs inside a database system I wonder how it is when it comes to ES. Will it kill performance or is it perfectly fine to store the information inside?
The idea here is to have the indexed information including the preview picture available from inside ES and then sync the index to his notebook so that he's able to search there when he's on the road.
Is there a way to make the indices on the notebook read only? I couldn't find it in the docs, but maybe I've just been blind.
If someone could point me into the right direction on these topics it'd be great.
Thanks a lot!