# Reducing heap size increases query speed

**URL:** <https://discuss.elastic.co/t/reducing-heap-size-increases-query-speed/311850>\
**Category:** Elasticsearch\
**Created:** [August 10, 2022, 2:58pm UTC](https://discuss.elastic.co/t/reducing-heap-size-increases-query-speed/311850 "2022-08-10T14:58:40Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![sbruinsje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sbruinsje/32/108568_2.png) [@sbruinsje](https://discuss.elastic.co/u/sbruinsje)\
**Post date:** [August 10, 2022, 2:58pm UTC](https://discuss.elastic.co/t/reducing-heap-size-increases-query-speed/311850/1 "2022-08-10T14:58:40Z")

</div>

I have recently upgraded from ES 6.8 to ES 8.3. One of the queries with heavy aggregations take significantly longer now compared to before. After some searching I found that in the new version ES sets the heap size automatically (to 1.8GB in my case), instead of the fixed default of 1GB in ES 6.8. It turns out that when I decrease the heap size back to 1GB the queries are faster again. I confirmed this by setting the heap size back to be determined automatically after which its significantly slower again.

The cluster is running on a single node with 4GB ram with 5 shards for the index. The index has a doc count of 18 million. Could it be that ES takes up so much heap space that there's not enough memory left for the filesystem cache? Or is there another explanation for why decreasing the heap size makes the queries slower.

For completeness here's the query we use:

```auto
POST article_set_index/_search
{
  "size": 0,
  "query": {
    "nested": {
      "path": "versions",
      "score_mode": "max",
      "inner_hits": {
        "size": 100,
        "highlight": {
          "pre_tags": [
            "<mark>"
          ],
          "post_tags": [
            "</mark>"
          ],
          "fields": {
            "versions.title": {
              "type": "fvh",
              "number_of_fragments": 0
            },
            "versions.body": {
              "type": "fvh",
              "number_of_fragments": 0
            }
          }
        },
        "sort": {
          "versions.published": {
            "order": "asc"
          }
        }
      },
      "query": {
        "bool": {
          "must": [
            {
              "simple_query_string": {
                "query": "the",
                "default_operator": "and",
                "fields": [
                  "versions.title",
                  "versions.body"
                ]
              }
            }
          ],
          "filter": [
            {
              "terms": {
                "versions.language": [
                  "nl"
                ]
              }
            },
            {
              "range": {
                "versions.created": {
                  "gte": 1644382800629
                }
              }
            },
            {
              "range": {
                "versions.publicationDate": {
                  "gte": "2022-02-09"
                }
              }
            }
          ]
        }
      }
    }
  },
  "aggs": {
    "totalAuthorCount": {
      "cardinality": {
        "field": "authorIds",
        "precision_threshold": 100
      }
    },
    "authors": {
      "terms": {
        "field": "authorIds",
        "size": 121,
        "shard_size": 400
      },
      "aggs": {
        "articles": {
          "top_hits": {
            "size": 5
          }
        }
      }
    }
  },
  "timeout": "60000ms"
}

```

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [August 10, 2022, 3:10pm UTC](https://discuss.elastic.co/t/reducing-heap-size-increases-query-speed/311850/2 "2022-08-10T15:10:21Z")

</div>

> [@sbruinsje](#):
>
> Could it be that ES takes up so much heap space that there's not enough memory left for the filesystem cache?

That would be my guess. What is the size of the index?

---

<div class="post-metadata">

**Author:** ![sbruinsje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sbruinsje/32/108568_2.png) [@sbruinsje](https://discuss.elastic.co/u/sbruinsje)\
**Post date:** [August 10, 2022, 3:18pm UTC](https://discuss.elastic.co/t/reducing-heap-size-increases-query-speed/311850/3 "2022-08-10T15:18:22Z")

</div>

About 38 GB:

```auto
GET /article_set_index/_stats?pretty
{
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "_all": {
    "primaries": {
      "docs": {
        "count": 18367658,
        "deleted": 583
      },
      "shard_stats": {
        "total_count": 5
      },
      "store": {
        "size_in_bytes": 38360064185,
        "total_data_set_size_in_bytes": 38360064185,
        "reserved_in_bytes": 0
      },
      ...
    }
  ...
  }
}

```

Side question: the docs count includes nested documents, not just root level documents, correct?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [August 10, 2022, 3:19pm UTC](https://discuss.elastic.co/t/reducing-heap-size-increases-query-speed/311850/4 "2022-08-10T15:19:44Z")

</div>

Yes, if you nave nested mappings it does.

As you are running aggregations and do not return specific documents I suspect there would be a small set of files that would be cached and where a small change in heap size could make a big difference.

---

<div class="post-metadata">

**Author:** ![sbruinsje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sbruinsje/32/108568_2.png) [@sbruinsje](https://discuss.elastic.co/u/sbruinsje)\
**Post date:** [August 10, 2022, 3:35pm UTC](https://discuss.elastic.co/t/reducing-heap-size-increases-query-speed/311850/5 "2022-08-10T15:35:12Z")

</div>

Hmm I don't think I follow. In my case decreasing the heap size makes the query with aggregations finish faster. If only a small set of files is being cached then why would decreasing heap size improve the speed?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [August 10, 2022, 3:56pm UTC](https://discuss.elastic.co/t/reducing-heap-size-increases-query-speed/311850/6 "2022-08-10T15:56:07Z")

</div>

I meant small compared to the full size of your index. If the amount of files that need to be cached is a few GB or so, decreasing the heap size would make a difference.

---

<div class="post-metadata">

**Author:** ![sbruinsje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sbruinsje/32/108568_2.png) [@sbruinsje](https://discuss.elastic.co/u/sbruinsje)\
**Post date:** [August 12, 2022, 8:43am UTC](https://discuss.elastic.co/t/reducing-heap-size-increases-query-speed/311850/7 "2022-08-12T08:43:30Z")

</div>

It turns out the difference in speed was not related to the heap size. I probably made some mistakes during measurements of the speed of the queries due to caching being involved.

Thanks for your help though 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 9, 2022, 8:43am UTC](https://discuss.elastic.co/t/reducing-heap-size-increases-query-speed/311850/8 "2022-09-09T08:43:42Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.