Best cache settings for static data in large index

I have a 180 million documents index (over 1 node and 1 shard), the index
is static I do not update it
I want to have best performance possible, I have 16 gigs of ram on my
machine

I have a intuition that there are some cache options that I can leverage.

Any ideas what might I need to configure

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

I would run an optimize on this index and set max_num_segments to 1:
http://www.elasticsearch.org/guide/reference/api/admin-indices-optimize/

This boost the performance of all search related operations.

if you use filters, I would also increase the size of the filter cache:
http://www.elasticsearch.org/guide/reference/index-modules/cache/

On 26 June 2013 13:49, David MZ david.mazvovsky@gmail.com wrote:

I have a 180 million documents index (over 1 node and 1 shard), the index
is static I do not update it
I want to have best performance possible, I have 16 gigs of ram on my
machine

I have a intuition that there are some cache options that I can leverage.

Any ideas what might I need to configure

Thanks

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Met vriendelijke groet,

Martijn van Groningen

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

If I have only 16gigs of ram, can I set more than 50% to elastic, I don't
have load at all, but once a query hits I want it to be fast
is larger heap size going to help me

I read that it's recommended to leave 50% to the OS, is that still true in
my scenario

On Wed, Jun 26, 2013 at 3:48 PM, Martijn v Groningen <
martijn.v.groningen@gmail.com> wrote:

I would run an optimize on this index and set max_num_segments to 1:
http://www.elasticsearch.org/guide/reference/api/admin-indices-optimize/

This boost the performance of all search related operations.

if you use filters, I would also increase the size of the filter cache:
http://www.elasticsearch.org/guide/reference/index-modules/cache/

On 26 June 2013 13:49, David MZ david.mazvovsky@gmail.com wrote:

I have a 180 million documents index (over 1 node and 1 shard), the index
is static I do not update it
I want to have best performance possible, I have 16 gigs of ram on my
machine

I have a intuition that there are some cache options that I can leverage.

Any ideas what might I need to configure

Thanks

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
Met vriendelijke groet,

Martijn van Groningen

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/VJQ0mQJy__s/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Yes that is still true in your scenario, the filesystem cache will
eventually use the unused ram, and make your queries in general faster.
How much of the heap is actually used? You can check this via the node
stats api (use the jvm option):
http://www.elasticsearch.org/guide/reference/api/admin-cluster-nodes-stats/

On 26 June 2013 14:55, David MZ david.mazvovsky@gmail.com wrote:

If I have only 16gigs of ram, can I set more than 50% to elastic, I don't
have load at all, but once a query hits I want it to be fast
is larger heap size going to help me

I read that it's recommended to leave 50% to the OS, is that still true in
my scenario

On Wed, Jun 26, 2013 at 3:48 PM, Martijn v Groningen <
martijn.v.groningen@gmail.com> wrote:

I would run an optimize on this index and set max_num_segments to 1:
http://www.elasticsearch.org/guide/reference/api/admin-indices-optimize/

This boost the performance of all search related operations.

if you use filters, I would also increase the size of the filter cache:
http://www.elasticsearch.org/guide/reference/index-modules/cache/

On 26 June 2013 13:49, David MZ david.mazvovsky@gmail.com wrote:

I have a 180 million documents index (over 1 node and 1 shard), the
index is static I do not update it
I want to have best performance possible, I have 16 gigs of ram on my
machine

I have a intuition that there are some cache options that I can leverage.

Any ideas what might I need to configure

Thanks

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
Met vriendelijke groet,

Martijn van Groningen

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/VJQ0mQJy__s/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Met vriendelijke groet,

Martijn van Groningen

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.