ElasticSearch + Rails in 1GB RAM VPS (CentOS)

Elastic Stack Elasticsearch
#1

Hello everyone,

Recently I've made an e-Commerce (a clone of eBay) app in Rails, and the search engine, I used the ElasticSearch (with the awesome Searchkick gem). In development machine, all works in perfectly harmony. (Vagrant - Ubuntu 14.04 - with 1 GB RAM)
In production machine, I've performed a default deploy procedures, like others deploys I've maded yet.
And I notice, sometimes my elasticsearch service dies. I have no idea what happen, to my ElasticSearch goes down.
My VPS is a CentOS with 1GB RAM, running a Nginx + Passenger to handle the Rails App. And has a Apache instance too, running to serve the WHMCS (but in 8080 port).
The average use of RAM machine is 640 MB RAM. I think is not a memory heap problem.
Anyone have idea?
Is the 1GB RAM to handle the NGINX + Passenger + ElasticSearch is insufficient?

#2

Its true that 1GB is suspiciously low for Elasticsearch. Especially with a Rails app etc all on the same box. If you read Elasticsearch The Definitive Guide you'll see the default heap size is 1GB -- too low

The default installation of Elasticsearch is configured with a 1 GB heap. For just about every deployment, this number is usually too small. If you are using the default heap values, your cluster is probably configured incorrectly.

You also need to leave RAM in reserve for the OS to cache the index files on disk. This is done outside the JVM.

Have you investigated your logs? /var/log/elasticsearch? Anything suspicious in there?

#3

Some days elasticsearch maintains working two days without dies. And some days the elasticsearch dies in same day.
In working time of elasticsearch, the app runs smooth and fast.
I don't know, but I think, I've made a mistake in ES setup.
What are recommended hardware specifications for ES?
I will look my log files again. :slight_smile:

#4

My gut says that a minimum ES setup for anything in production would be an 8 GB linux box with 4GB given to the JVM and 4GB left in reserve for the OS cache. That's a starting point, and there's so many directions you could go from there if further tuning or caching is needed.

#5

I noticed the respective behaviour:
(Example) Actual RAM usage -> 614MB
When I made a search, ES is called by Rails App and the RAM grows to 645 MB, and maintains in this value indefinitely. If a made another search in few minutes later, the RAM grows again, and in every search grows the RAM usage. And never release memory. In the end, the RAM usage gets 1024, and ES dies.
It's my first time using ES, I'm sorry if my doubts is so obvious for you guys, but ES is a new challenge for me. And I like the ES way to work. :slight_smile:

#6