Yesterday i thought there was with my small vm (2gb ram, 2gb disk) and a
million items in my es database, So I started dropping types in the same
index, in the end dropped 70k types and did not solve the issue (something
else was the issue), none the less elasticsearch paid me a visit with
merging documents i deleted out. This took over 12 hours and it was doing
it in bulk and when it did them CPU went up for a while 1-5 minutes. Is
this suppopse to happen or is merge suppose to be a little more gentail.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Hello,
I guess merging can be painfull if it's caused by massive deletes on a
small vm, because you can get up to 30 going on at the same time, if my
understanding is correct (index.merge.policy.max_merge_at_once_explicit is
30 by default).
You should be able to make it more gentle by reducing that number though
from the index settingshttp://www.elasticsearch.org/guide/reference/api/admin-indices-update-settings/
.
Also, if your CPU load is caused by I/O waits, store
throttlinghttp://www.elasticsearch.org/guide/reference/index-modules/store/might
also help.
Best regards,
Radu
http://sematext.com/ -- Elasticsearch -- Solr -- Lucene
On Sat, May 4, 2013 at 8:28 AM, Wojons Tech wojonstech@gmail.com wrote:
Yesterday i thought there was with my small vm (2gb ram, 2gb disk) and a
million items in my es database, So I started dropping types in the same
index, in the end dropped 70k types and did not solve the issue (something
else was the issue), none the less elasticsearch paid me a visit with
merging documents i deleted out. This took over 12 hours and it was doing
it in bulk and when it did them CPU went up for a while 1-5 minutes. Is
this suppopse to happen or is merge suppose to be a little more gentail.
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Okay so I will look at lowering that number to something resonable for the
size of my vm. Also the link to store throttling is very intresting and i
am going to try that out. The goal is to have indexes by date and alias
them but for dev i am just storing everything in the same one until i write
the code to handle that.
On Sat, May 4, 2013 at 12:52 AM, Radu Gheorghe
radu.gheorghe@sematext.comwrote:
Hello,
I guess merging can be painfull if it's caused by massive deletes on a
small vm, because you can get up to 30 going on at the same time, if my
understanding is correct (index.merge.policy.max_merge_at_once_explicit is
30 by default).
You should be able to make it more gentle by reducing that number though
from the index settingshttp://www.elasticsearch.org/guide/reference/api/admin-indices-update-settings/
.
Also, if your CPU load is caused by I/O waits, store throttlinghttp://www.elasticsearch.org/guide/reference/index-modules/store/might also help.
Best regards,
Radu
http://sematext.com/ -- Elasticsearch -- Solr -- Lucene
On Sat, May 4, 2013 at 8:28 AM, Wojons Tech wojonstech@gmail.com wrote:
Yesterday i thought there was with my small vm (2gb ram, 2gb disk) and a
million items in my es database, So I started dropping types in the same
index, in the end dropped 70k types and did not solve the issue (something
else was the issue), none the less elasticsearch paid me a visit with
merging documents i deleted out. This took over 12 hours and it was doing
it in bulk and when it did them CPU went up for a while 1-5 minutes. Is
this suppopse to happen or is merge suppose to be a little more gentail.
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/1fyUmiqQzWc/unsubscribe?hl=en-US
.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
--
Enjoy,
Alexis Okuwa
WojonsTech
424.835.1223
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.