Fielddata cache sanity check

Elastic Stack Elasticsearch
1

Hello all,

Just want to make sure my field data settings are going to work in a way
that will protect my cluster from a bad query.

elasticsearch.yml
indices.fielddata.cache.size: 11GB

get/_cluster/settings
{
"persistent": {
"indices": {
"cache": {
"filter": {
"expire": "48h"
}
},
"breaker": {
"total": {
"limit": "15GB"
},
"fielddata": {
"limit": "12GB"
}
},
"fielddata": {
"breaker": {
"limit": "80%"
}
}
}
}
}

Does everything look right here? I tried to get rid of
indices.fielddata.breaker.limit but it just remains.

Thanks!

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/4a2073f5-df80-46ed-abdb-7beca2681f2a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

Can anybody comment on this? Does this look ok?

Thanks,

Drew

On Wednesday, March 11, 2015 at 3:02:56 PM UTC-5, Drew Town wrote:

Hello all,

Just want to make sure my field data settings are going to work in a way
that will protect my cluster from a bad query.

elasticsearch.yml
indices.fielddata.cache.size: 11GB

get/_cluster/settings
{
"persistent": {
"indices": {
"cache": {
"filter": {
"expire": "48h"
}
},
"breaker": {
"total": {
"limit": "15GB"
},
"fielddata": {
"limit": "12GB"
}
},
"fielddata": {
"breaker": {
"limit": "80%"
}
}
}
}
}

Does everything look right here? I tried to get rid of
indices.fielddata.breaker.limit but it just remains.

Thanks!

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/11d056b3-60e7-450b-b0ff-f4c95717ca7b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

3

It's not really worth flushing things from cache based on time,
Elasticsearch uses LRU (last recently used) to clear things and it's best
to just let that handle it.

Don't forget that allowing that much fielddata in your heap means there is
less for actual good things to happen, as the JVM cannot GC this data from
the heap. If you are seeing a lot of field data then you're probably better
off leveraging doc values.

On 19 March 2015 at 06:08, Drew Town drew.town@gmail.com wrote:

Can anybody comment on this? Does this look ok?

Thanks,

Drew

On Wednesday, March 11, 2015 at 3:02:56 PM UTC-5, Drew Town wrote:

Hello all,

Just want to make sure my field data settings are going to work in a way
that will protect my cluster from a bad query.

elasticsearch.yml
indices.fielddata.cache.size: 11GB

get/_cluster/settings
{
"persistent": {
"indices": {
"cache": {
"filter": {
"expire": "48h"
}
},
"breaker": {
"total": {
"limit": "15GB"
},
"fielddata": {
"limit": "12GB"
}
},
"fielddata": {
"breaker": {
"limit": "80%"
}
}
}
}
}

Does everything look right here? I tried to get rid of
indices.fielddata.breaker.limit but it just remains.

Thanks!

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/11d056b3-60e7-450b-b0ff-f4c95717ca7b%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/11d056b3-60e7-450b-b0ff-f4c95717ca7b%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_NJvJn9STGqTEz%3DZth3HyKL250DDHfbkhCv3Uy7pRUYA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

4

Mark,

Thanks for the idea of using doc values. I added our @timestamp field to
doc values and it has made a huge impact on reducing our field data cache
size with only a pretty minimal impact on performance. It has worked so
well I may actually add some more commonly used fields to doc values.

Thanks,

Drew

On Thursday, March 19, 2015 at 11:45:26 AM UTC-5, Mark Walkom wrote:

It's not really worth flushing things from cache based on time,
Elasticsearch uses LRU (last recently used) to clear things and it's best
to just let that handle it.

Don't forget that allowing that much fielddata in your heap means there is
less for actual good things to happen, as the JVM cannot GC this data from
the heap. If you are seeing a lot of field data then you're probably better
off leveraging doc values.

On 19 March 2015 at 06:08, Drew Town <drew...@gmail.com <javascript:>>
wrote:

Can anybody comment on this? Does this look ok?

Thanks,

Drew

On Wednesday, March 11, 2015 at 3:02:56 PM UTC-5, Drew Town wrote:

Hello all,

Just want to make sure my field data settings are going to work in a way
that will protect my cluster from a bad query.

elasticsearch.yml
indices.fielddata.cache.size: 11GB

get/_cluster/settings
{
"persistent": {
"indices": {
"cache": {
"filter": {
"expire": "48h"
}
},
"breaker": {
"total": {
"limit": "15GB"
},
"fielddata": {
"limit": "12GB"
}
},
"fielddata": {
"breaker": {
"limit": "80%"
}
}
}
}
}

Does everything look right here? I tried to get rid of
indices.fielddata.breaker.limit but it just remains.

Thanks!

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/11d056b3-60e7-450b-b0ff-f4c95717ca7b%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/11d056b3-60e7-450b-b0ff-f4c95717ca7b%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/97ca2390-5944-4b9b-90cb-ddf595b46293%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.