Use Pogo Plug computers dirt cheap, high node clusters?

Elastic Stack Elasticsearch
1

I'm been playing around with this -- more for fun than profit -- but I
thought I would share and get some ideas.

I have 3 Plug computers (Seagate Dockstar) that I bought for around
$30 a piece. These are cool full mini-Linux servers where I've
installed Arch Linux and OpenSDK. Each I have attached a cheap 500GB
drive that ran me about $60. So my per-server cost (with tax) is just
over $100.

Last night I loaded ElasticSearch on each one and put them into a
cluster. These boxes don't have much RAM (128MB), so I dedicated 100MB
to each Elastic instance and it seems to run pretty well with this
simple configuration.

Although this is just play right now, it struck me that you might be
able to create a cheap, high node cluster with Dockstars.

Maybe you could configure them as "node.data:true" servers to store
all the data, create the index with a high number of shards, and then
have one normal machine with decent RAM be the "node.master: false" /
"node.data: false" search load balancer to aggregate result sets.

Is this too little RAM per node to allow Elastic to function
effectively? Are there some other systemic problems that would make
this simply a waste of time?

Please let me know what you think!

2

On Dec 26 2011, 6:30 pm, danr drahm...@gmail.com wrote:> ... Last
night I loaded Elasticsearch on each one and put them into a

cluster. These boxes don't have much RAM (128MB), so I dedicated 100MB
to each Elastic instance and it seems to run pretty well with this
simple configuration.
...
Although this is just play right now, it struck me that you might be
able to create a cheap, high node cluster with Dockstars.

Maybe you could configure them as "node.data:true" servers to store
all the data, create the index with a high number of shards, and then
have one normal machine with decent RAM be the "node.master: false" /
"node.data: false" search load balancer to aggregate result sets.

Is this too little RAM per node to allow Elastic to function
effectively? Are there some other systemic problems that would make
this simply a waste of time?

You'll definitely want to test. Cool idea, but I'm not optimistic that
elasticsearch will do well with such little memory, even if you have a
lot of nodes.

Sorry I don't have numbers, but I tested some low-memory nodes and
kept getting OutOfMemoryException's at roughly the same number of
documents indexed, even after adding nodes and increasing shards.

--Jamshid