we indexed 250 million docs on 6 es nodes. es's fast index speed and
relatively low resource (both cpu and memory) requirement is very
good. the average index-speed for 1000 docs is about 100ms.
we did several performance tests, and they were all passed. but when
we reindex all the docs (delete and recreate index & mappings before
that), it becomes quite slow. about 1600ms for 1000 docs on average.
so we restart all the nodes, and the index speed become 100ms again.
i guess it's the high memory consumption that hurts the index speed.
because bigdesk shows that, about 10g virtual memory were taken by es
on every node. and it seems that they were not reclaimed by gc.
so what can i do to make index speed fast without restarting nodes?
delete & recreate index won't help, as i already did that.
ps, my environment:
es version:0.19.9 with 6 shards and no replica.
jdk: 1.6u25 (64 bit)
memory: 5g(physical) + 2g(swap)
index file size:60g(most fields are not stored)
JAVA_OPTS='-Xmx3500m -Xms3500m -Xmn768m -XX:PermSize=128m -
XX:MaxPermSize=128m -Xss256k -XX:+DisableExplicitGC'