Seeing degradation of repeated search queries over time

Elastic Stack Elasticsearch
1

We are observing a rather strange behavior. The same query keeps getting slower with every subsequent request without us loading up any new data. After ES server has been running for a couple of days, the speed goes down to 10% of what it was right after reboot.

2

What version? What JVM? What are your settings?
What is the query?

There's not much to work with here, so more details would help us help you :slight_smile:

3

Elastic 5.0.2
JVM: java-1.8.0-openjdk
JVM options: -Xmx80g -Xms80g

non-default settings in elasticsearch.yml:

script.inline: true
script.engine.groovy.inline.aggs: true
script.engine.groovy.inline.update: true
script.engine.groovy.inline.search: true
action.auto_create_index: true
4

That is a very large heap - much bigger than what we recommend. Do you have monitoring installed so you can look at how heap usage and GC varies over time? Do you see anything around GC reported in the logs?

As you also seem to be using inline Groovy scripts I did a quick search on Github. I found a few issues (this and this) around not using scripts with parameters which can cause memory problems as scripts are constantly recompiled. How are you using the inline Groovy scripts?

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.