# Field cache efficiency
**URL:** https://discuss.elastic.co/t/field-cache-efficiency/7269
**Category:** Elasticsearch
**Created:** [April 9, 2012, 11:50am UTC](https://discuss.elastic.co/t/field-cache-efficiency/7269 "2012-04-09T11:50:54Z")
**Posts on this page:** 5
**Page:** 1
### Author:  [@Christos\_Trochalakis](https://discuss.elastic.co/u/Christos_Trochalakis)
#### Post date: [April 9, 2012, 11:50am UTC](https://discuss.elastic.co/t/field-cache-efficiency/7269/1 "2012-04-09T11:50:54Z")
Hello, I'd like to ask a question about field caching on fields not
present on all documents.
We maintain a flat index with documents belonging to different
categories, our documents are like this
{
name: "Document name",
category\_id: 42,
filters: {
fg\_10: [10,20,30,40],
fg\_11: [100, 200, 300, 400]
}
}
- Each document, depending on its category\_id, has different filter
keys, for example documents with category\_id = 42 have:
filters: {
fg\_10: [],
fg\_11: []
}
where documents with category\_id = 25 have:
filters: {
fg\_20: [],
fg\_21: []
}
One of our two main queries is a term filtered query for a specific
category, with faceting on that category fg\_\* keys (filters.fg\_10,
filters.fg\_11)
According to what I have read, elastic keeps a field cache for
each faceted field, so in our case there should be a field cache for
fg\_10, fg\_11, fg\_20, fg\_21)
I'd like to know how space-efficient the field cache for our setup is.
For example, is the field cache for fg\_10 "smart" enough to only
include documents for category\_id=40 or it includes all index
documents taking more space?
---
### Author:  [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [April 11, 2012, 9:33am UTC](https://discuss.elastic.co/t/field-cache-efficiency/7269/2 "2012-04-11T09:33:31Z")
There will be an overhead per field, which is the array of integers (sized
as the number of documents (in each segment)) for each field. How many
"fg's" do you have?
On Mon, Apr 9, 2012 at 2:50 PM, Christos Trochalakis [christos@skroutz.gr](mailto:christos@skroutz.gr)wrote:
> Hello, I'd like to ask a question about field caching on fields not
> present on all documents.
>
> We maintain a flat index with documents belonging to different
> categories, our documents are like this
>
> {
> name: "Document name",
> category\_id: 42,
> filters: {
> fg\_10: [10,20,30,40],
> fg\_11: [100, 200, 300, 400]
> }
> }
>
> - Each document, depending on its category\_id, has different filter
> keys, for example documents with category\_id = 42 have:
>
> filters: {
> fg\_10: [],
> fg\_11: []
> }
>
> where documents with category\_id = 25 have:
>
> filters: {
> fg\_20: [],
> fg\_21: []
> }
>
> One of our two main queries is a term filtered query for a specific
> category, with faceting on that category fg\_\* keys (filters.fg\_10,
> filters.fg\_11)
>
> According to what I have read, elastic keeps a field cache for
> each faceted field, so in our case there should be a field cache for
> fg\_10, fg\_11, fg\_20, fg\_21)
>
> I'd like to know how space-efficient the field cache for our setup is.
> For example, is the field cache for fg\_10 "smart" enough to only
> include documents for category\_id=40 or it includes all index
> documents taking more space?
---
### Author:  [@Christos\_Trochalakis](https://discuss.elastic.co/u/Christos_Trochalakis)
#### Post date: [April 11, 2012, 12:18pm UTC](https://discuss.elastic.co/t/field-cache-efficiency/7269/3 "2012-04-11T12:18:11Z")
Thanks for the reply, Shay
We have 1.400.000 documents, spanning on ~900 different
categories. We have 1.200 different filter groups (so ~1.3
fgs/category on average).
On Wed, 11 Apr 2012 12:33:31 +0300
Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote:
> There will be an overhead per field, which is the array of integers
> (sized as the number of documents (in each segment)) for each field.
> How many "fg's" do you have?
>
> On Mon, Apr 9, 2012 at 2:50 PM, Christos Trochalakis
> [christos@skroutz.gr](mailto:christos@skroutz.gr)wrote:
>
> > Hello, I'd like to ask a question about field caching on fields not
> > present on all documents.
> >
> > We maintain a flat index with documents belonging to different
> > categories, our documents are like this
> >
> > {
> > name: "Document name",
> > category\_id: 42,
> > filters: {
> > fg\_10: [10,20,30,40],
> > fg\_11: [100, 200, 300, 400]
> > }
> > }
> >
> > - Each document, depending on its category\_id, has different filter
> > keys, for example documents with category\_id = 42 have:
> >
> > filters: {
> > fg\_10: [],
> > fg\_11: []
> > }
> >
> > where documents with category\_id = 25 have:
> >
> > filters: {
> > fg\_20: [],
> > fg\_21: []
> > }
> >
> > One of our two main queries is a term filtered query for a specific
> > category, with faceting on that category fg\_\* keys (filters.fg\_10,
> > filters.fg\_11)
> >
> > According to what I have read, elastic keeps a field cache for
> > each faceted field, so in our case there should be a field cache for
> > fg\_10, fg\_11, fg\_20, fg\_21)
> >
> > I'd like to know how space-efficient the field cache for our setup
> > is. For example, is the field cache for fg\_10 "smart" enough to only
> > include documents for category\_id=40 or it includes all index
> > documents taking more space?
---
### Author:  [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [April 13, 2012, 12:07pm UTC](https://discuss.elastic.co/t/field-cache-efficiency/7269/4 "2012-04-13T12:07:24Z")
I see. It will use memory... and come with more overhead, I suggest you run
your tests and see how much memory the field cache is using through the
node stats.
On Wed, Apr 11, 2012 at 3:18 PM, Christos Trochalakis
[christos@skroutz.gr](mailto:christos@skroutz.gr)wrote:
> Thanks for the reply, Shay
>
> We have 1.400.000 documents, spanning on ~900 different
> categories. We have 1.200 different filter groups (so ~1.3
> fgs/category on average).
>
> On Wed, 11 Apr 2012 12:33:31 +0300
> Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote:
>
> > There will be an overhead per field, which is the array of integers
> > (sized as the number of documents (in each segment)) for each field.
> > How many "fg's" do you have?
> >
> > On Mon, Apr 9, 2012 at 2:50 PM, Christos Trochalakis
> > [christos@skroutz.gr](mailto:christos@skroutz.gr)wrote:
> >
> > > Hello, I'd like to ask a question about field caching on fields not
> > > present on all documents.
> > >
> > > We maintain a flat index with documents belonging to different
> > > categories, our documents are like this
> > >
> > > {
> > > name: "Document name",
> > > category\_id: 42,
> > > filters: {
> > > fg\_10: [10,20,30,40],
> > > fg\_11: [100, 200, 300, 400]
> > > }
> > > }
> > >
> > > - Each document, depending on its category\_id, has different filter
> > > keys, for example documents with category\_id = 42 have:
> > >
> > > filters: {
> > > fg\_10: [],
> > > fg\_11: []
> > > }
> > >
> > > where documents with category\_id = 25 have:
> > >
> > > filters: {
> > > fg\_20: [],
> > > fg\_21: []
> > > }
> > >
> > > One of our two main queries is a term filtered query for a specific
> > > category, with faceting on that category fg\_\* keys (filters.fg\_10,
> > > filters.fg\_11)
> > >
> > > According to what I have read, elastic keeps a field cache for
> > > each faceted field, so in our case there should be a field cache for
> > > fg\_10, fg\_11, fg\_20, fg\_21)
> > >
> > > I'd like to know how space-efficient the field cache for our setup
> > > is. For example, is the field cache for fg\_10 "smart" enough to only
> > > include documents for category\_id=40 or it includes all index
> > > documents taking more space?
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 3:32am UTC](https://discuss.elastic.co/t/field-cache-efficiency/7269/5 "2017-07-06T03:32:44Z")