If you have just a single node, and don't plan to expand the size of the
index, then a single shard is enough. As to replicas, if you have a single
node, then no replicas is fine (though even with 1 replica, that replica is
not allocated on the same node), and you can always increase it if you want.
A single shard will use less memory than 3 shards.
On Wednesday, May 4, 2011 at 11:00 AM, Enrique Medina Montenegro wrote:
Maybe I'm overconfiguring the ES server, as I only need it to search
products by two fields. I used the defaults: 3 shards and 1 replica, but as
long as this data is not critical whatsoever (I could regenerate it in some
hours, and it doesn't stop my business), maybe I should just give it 1 shard
and no replicas.
What do you think could be the minimal configuration for ES to work as per
On Tue, May 3, 2011 at 11:12 PM, Shay Banon firstname.lastname@example.org:
They don't take 5 minutes each, they take 5 second, the first 
indicates the time it took, while the second / indicates the total time.
In any case, it seems like you might need to assign more memory for this
cluster nodes. Node stats can give you a start in trying to understand where
the memory is going.
On Tuesday, May 3, 2011 at 9:42 PM, Enrique Medina Montenegro wrote:
Today I noticed much more activity that any other days in terms of GC:
[2011-05-02 12:20:17,327][INFO ][monitor.jvm ] [Uni-Mind]
[gc][ConcurrentMarkSweep] took [7.9s]/[3.7m], reclaimed [17.7mb],
leaving [612.9mb] used, max [1gb]
[2011-05-02 12:21:02,395][INFO ][monitor.jvm ] [Uni-Mind]
[gc][ConcurrentMarkSweep] took [5s]/[4.2m], reclaimed [8.4mb], leaving
[842.3mb] used, max [1gb]
[2011-05-02 12:21:16,026][INFO ][monitor.jvm ] [Uni-Mind]
[gc][ConcurrentMarkSweep] took [5.7s]/[4.4m], reclaimed [14.9mb],
leaving [874.7mb] used, max [1gb]
[2011-05-02 12:22:12,922][INFO ][monitor.jvm ] [Uni-Mind]
[gc][ConcurrentMarkSweep] took [5s]/[5m], reclaimed [20.1mb], leaving
[673mb] used, max [1gb]
[2011-05-02 12:22:19,094][INFO ][monitor.jvm ] [Uni-Mind]
[gc][ConcurrentMarkSweep] took [5.4s]/[5.1m], reclaimed [18.1mb],
leaving [700.2mb] used, max [1gb]
[2011-05-02 12:22:26,582][INFO ][monitor.jvm ] [Uni-Mind]
[gc][ConcurrentMarkSweep] took [5s]/[5.2m], reclaimed [19mb], leaving
[718.5mb] used, max [1gb]
[2011-05-02 12:22:35,417][INFO ][monitor.jvm ] [Uni-Mind]
[gc][ConcurrentMarkSweep] took [5.1s]/[5.3m], reclaimed [14.6mb],
leaving [745.1mb] used, max [1gb]
Does it make sense to perform 7 consecutive GCs? And lasting an avergae of
5 minutes each? It's a bit scary...
On Tue, Apr 19, 2011 at 11:04 AM, Shay Banon <email@example.com
I have seen some dubious messages regarding GC that ES logs lately which
seems to indicate that the API ES uses to try and log long GCs has either
changed or got broken (it uses an internal JVM API to get it). Need to look
On Monday, April 18, 2011 at 1:00 PM, Enrique Medina Montenegro wrote:
Thanks a lot for your interest in my GC issues...
Out of your list, I just added:
to my elasticsearch.yml config file (hope the format if fine in YML),
because I already fulfilled the other hints you were suggesting:
ulimit -l unlimited
/var/elasticsearch-0.16.0-SNAPSHOT/bin/elasticsearch -f -Xmx1g -Xms1g &
Just for your evaluation, these are some samples of my GC output:
[2011-04-18 10:43:12,001][INFO ][monitor.jvm ] [Earthquake]
[gc][ConcurrentMarkSweep] took [27.5s]/[27.7s], reclaimed [532.8mb],
leaving [255.4mb] used, max [1.1gb]
But FYI, I've seen sometimes times in minutes...
On Mon, Apr 18, 2011 at 11:28 AM, Clinton Gormley <firstname.lastname@example.org
I also have 0.16 working on my PROD env without issues, other than
some GC delays...
How long are these delays? Do they cause ES to pause?
It's quite normal to get GC messages in the logs, eg:
[gc][ConcurrentMarkSweep] took [12.2s]/[2.9s], reclaimed [3.2gb],
leaving [7.1gb] used, max [14.1gb]
[gc][ConcurrentMarkSweep] took [10s]/[3s], reclaimed [2.7gb], leaving
[7.6gb] used, max [14.1gb]
However, if any of your Java heap is sitting in swap, it can really
affect performance, and it is worth doing some work to make sure that
you never touch the swap.
Linux can put some data into swap even when there is plenty of free
You may already be doing this, but just in case, here is how I would set
If you have eg 8GB of memory available on your box to run ES, I would
use 50-60% of that for ES itself, and leave the rest for kernel
filesystem caches etc
Set your ES_MIN_MEM and ES_MAX_MEM to the same value (eg 4GB)
Set "ulimit -l unlimited" before starting ES - this gives the ES
process permission to lock memory.
I run ES as user 'elasticsearch' and I have this line in
elasticsearch - memlock unlimited
- Add "boostrap.mlockall: 1" to your config file, to tell ES to lock
If ES is the only service you have running on the box, you may just want
to set swapoff -a
For more, see:
Note: Ubuntu doesn't read /etc/security/limits.conf by default, when
su'ing to a different user. To enabled that, you have to uncomment this
line in /etc/pam.d/su:
session required pam_limits.so