# Node tasks and their memory requirements **URL:** \ **Category:** Elasticsearch\ **Created:** [October 11, 2013, 11:15am UTC](https://discuss.elastic.co/t/node-tasks-and-their-memory-requirements/13923 "2013-10-11T11:15:52Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![Taylor\_Fort](https://avatars.discourse-cdn.com/v4/letter/t/8491ac/32.png) [@Taylor\_Fort](https://discuss.elastic.co/u/Taylor_Fort)\ **Post date:** [October 11, 2013, 11:15am UTC](https://discuss.elastic.co/t/node-tasks-and-their-memory-requirements/13923/1 "2013-10-11T11:15:52Z")
we have an 18 node setup with a single replica, 18 shard index. When we test out traffic against the ES cluster, we notice gradual jumps in the filter\_cache\_size until it hits the memory threshold and then we start to see filter cache evictions (~10 per query). Currently, we have nothing setup in our configs that specifies a node task, so every node is handling queries, searches, and indexing, but we're noticing nodes will start becoming unresponsive upon the GC heavily cleaning up memory. Note: we heavily use facets and boolean filters (nature of our query requirements). I'm referencing this guide page for node tasks... [http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-node.html](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-node.html) Questions... 1. what is the memory impact on a node that is handling only queries (i.e. if I set it to data : false)? 2. is the filter cache size primarily related to the node that does the scatter and gather search, the node that has the actual data, or both? is adding a replica on a data node essentially adding n + 1 requirements for my filter cache size? Thanks -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [October 14, 2013, 7:43am UTC](https://discuss.elastic.co/t/node-tasks-and-their-memory-requirements/13923/2 "2013-10-14T07:43:20Z")
Hey, to your questions (and hopefully a bit beyond) 1. A non-data node (so a master node or a client node) is handling the reduce phase during a search. So if you search over 5 shards, you will get back a certain amount of results from each shard, which are then reduced on the node, where the search originated from. So if you do deep paging (which you should not do), you could also put a non-data node under memory pressure. Apart from that memory requirements are much lower on these nodes as they do not need fielddata or filter caches. 2. Filter caches are per lucene segment and lucene segments require data to be stored. Do you monitor your system? Can you identify the reasons for the long GC runs (I guess it is most likely fielddata, so you should monitor the size for each field you are faceting on, you might want to filter by frequency or regex to decrease data loaded into memory). You can also increase the filter cache size if you think it is not enough, see > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. Hope this helps. Otherwise feel free to ask for more details 🙂 --Alex On Fri, Oct 11, 2013 at 1:15 PM, Taylor Fort [taylor.fort@gmail.com](mailto:taylor.fort@gmail.com) wrote: > we have an 18 node setup with a single replica, 18 shard index. When we > test out traffic against the ES cluster, we notice gradual jumps in the > filter\_cache\_size until it hits the memory threshold and then we start to > see filter cache evictions (~10 per query). Currently, we have nothing > setup in our configs that specifies a node task, so every node is handling > queries, searches, and indexing, but we're noticing nodes will start > becoming unresponsive upon the GC heavily cleaning up memory. Note: we > heavily use facets and boolean filters (nature of our query requirements). > > I'm referencing this guide page for node tasks... > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-node.html) > > Questions... > > 1. what is the memory impact on a node that is handling only queries (i.e. > if I set it to data : false)? > > 2. is the filter cache size primarily related to the node that does the > scatter and gather search, the node that has the actual data, or both? is > adding a replica on a data node essentially adding n + 1 requirements for > my filter cache size? > > Thanks > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![otisg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/otisg/32/492_2.png) [@otisg](https://discuss.elastic.co/u/otisg)\ **Post date:** [October 15, 2013, 4:13am UTC](https://discuss.elastic.co/t/node-tasks-and-their-memory-requirements/13923/3 "2013-10-15T04:13:43Z")
Hi, To add a note to Alex's mention of system monitoring. You may want to peek at the JVM Memory report in SPM which will tell you what's going on with different memory pools. Stick that on a Dashboard next to the GC graph and you should be able to see correlation pretty clearly and that will tell you which pool you could make bigger. If you want, you can email your graphs to this list _directly_ from SPM - look for the little ambulance icon above graphs. We've been using G1 a lot lately. See [How to Tune Java Garbage Collection - Sematext](http://blog.sematext.com/2013/06/24/g1-cms-java-garbage-collector/) ## Otis ELASTICSEARCH Performance Monitoring - [Sematext Monitoring | Infrastructure Monitoring Service](http://sematext.com/spm/index.html) Search Analytics - [Cloud Monitoring Tools & Services | Sematext](http://sematext.com/search-analytics/index.html) On Monday, October 14, 2013 3:43:20 AM UTC-4, Alexander Reelsen wrote: > Hey, > > to your questions (and hopefully a bit beyond) > > 1. A non-data node (so a master node or a client node) is handling the > reduce phase during a search. So if you search over 5 shards, you will get > back a certain amount of results from each shard, which are then reduced on > the node, where the search originated from. So if you do deep paging (which > you should not do), you could also put a non-data node under memory > pressure. Apart from that memory requirements are much lower on these nodes > as they do not need fielddata or filter caches. > > 2. Filter caches are per lucene segment and lucene segments require data > to be stored. > > Do you monitor your system? Can you identify the reasons for the long GC > runs (I guess it is most likely fielddata, so you should monitor the size > for each field you are faceting on, you might want to filter by frequency > or regex to decrease data loaded into memory). You can also increase the > filter cache size if you think it is not enough, see > > [Elastic — The Search AI Company | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-nodes-stats.html#field-data) > > [Elastic — The Search AI Company | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-core-types.html#fielddata-filters) > > [Elastic — The Search AI Company | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/index-modules-fielddata.html#_filtering_by_frequency) > > [Elastic — The Search AI Company | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/index-modules-cache.html#filter) > > Hope this helps. Otherwise feel free to ask for more details 🙂 > > --Alex > > On Fri, Oct 11, 2013 at 1:15 PM, Taylor Fort \<[taylo...@gmail.com](mailto:taylo...@gmail.com)\ > > > wrote: > > > we have an 18 node setup with a single replica, 18 shard index. When we > > test out traffic against the ES cluster, we notice gradual jumps in the > > filter\_cache\_size until it hits the memory threshold and then we start to > > see filter cache evictions (~10 per query). Currently, we have nothing > > setup in our configs that specifies a node task, so every node is handling > > queries, searches, and indexing, but we're noticing nodes will start > > becoming unresponsive upon the GC heavily cleaning up memory. Note: we > > heavily use facets and boolean filters (nature of our query requirements). > > > > I'm referencing this guide page for node tasks... > > [Elastic — The Search AI Company | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-node.html) > > > > Questions... > > > > 1. what is the memory impact on a node that is handling only queries > > (i.e. if I set it to data : false)? > > > > 2. is the filter cache size primarily related to the node that does the > > scatter and gather search, the node that has the actual data, or both? is > > adding a replica on a data node essentially adding n + 1 requirements for > > my filter cache size? > > > > Thanks > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:12am UTC](https://discuss.elastic.co/t/node-tasks-and-their-memory-requirements/13923/4 "2017-07-06T02:12:26Z")
Closed on July 6, 2017, 2:12am UTC