ElasticSearch works very slow

Elastic Stack Elasticsearch
1

Suddenly, my ElasticSearch works very slow. When I do a query, I get the following:

[2018-10-16T11:33:41,924][ERROR][o.e.x.s.a.f.SecurityActionFilter] [OGAyjwj] blocking [cluster:monitor/stats] operation due to expired license. Cluster health, cluster stats and indices stats
operations are blocked on license expiration. All data operations (read and write) continue to work.
If you have a new license, please update it. Otherwise, please reach out to your support contact.
[2018-10-16T11:37:48,449][ERROR][o.e.x.w.t.s.ExecutableScriptTransform] [OGAyjwj] failed to execute [script] transform for [ilnu0qQESvGSag2o4e3pJw_elasticsearch_cluster_status_14c0137e-b0e1-4130-b965-fdc5da1d8e4d-2018-10-16T09:37:48.436Z]
org.elasticsearch.script.ScriptException: runtime error
        at org.elasticsearch.painless.PainlessScript.convertToScriptException(PainlessScript.java:101) ~[?:?]
        at org.elasticsearch.painless.PainlessScript$Script.execute(ctx.vars.email_recipient = (ctx.payload.kibana_settings.hits.total > 0) ? ctx.payload.kibana_settings.hits.hits[0]._source.kibana_settings.xpack.default_admin_email : null;ctx.vars.is_new = ctx.vars.fails_check && !ctx.vars.not_resolved;ctx.vars.is_resolve ...:1070) ~[?:?]
        at org.elasticsearch.painless.ScriptImpl.run(ScriptImpl.java:105) ~[?:?]
        at org.elasticsearch.xpack.watcher.transform.script.ExecutableScriptTransform.doExecute(ExecutableScriptTransform.java:69) ~[x-pack-watcher-6.2.3.jar:6.2.3]
        at org.elasticsearch.xpack.watcher.transform.script.ExecutableScriptTransform.execute(ExecutableScriptTransform.java:53) ~[x-pack-watcher-6.2.3.jar:6.2.3]
        at org.elasticsearch.xpack.watcher.transform.script.ExecutableScriptTransform.execute(ExecutableScriptTransform.java:38) ~[x-pack-watcher-6.2.3.jar:6.2.3]
        at org.elasticsearch.xpack.watcher.execution.ExecutionService.executeInner(ExecutionService.java:481) ~[x-pack-watcher-6.2.3.jar:6.2.3]
        at org.elasticsearch.xpack.watcher.execution.ExecutionService.execute(ExecutionService.java:322) ~[x-pack-watcher-6.2.3.jar:6.2.3]
        at org.elasticsearch.xpack.watcher.execution.ExecutionService.lambda$executeAsync$7(ExecutionService.java:426) ~[x-pack-watcher-6.2.3.jar:6.2.3]
        at org.elasticsearch.xpack.watcher.execution.ExecutionService$WatchExecutionTask.run(ExecutionService.java:580) [x-pack-watcher-6.2.3.jar:6.2.3]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:573) [elasticsearch-6.2.3.jar:6.2.3]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:1.8.0_111]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:1.8.0_111]
        at java.lang.Thread.run(Unknown Source) [?:1.8.0_111]
Caused by: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
        at java.util.ArrayList.rangeCheck(Unknown Source) ~[?:1.8.0_111]
        at java.util.ArrayList.get(Unknown Source) ~[?:1.8.0_111]
        at org.elasticsearch.painless.PainlessScript$Script.execute(ctx.vars.email_recipient = (ctx.payload.kibana_settings.hits.total > 0) ? ctx.payload.kibana_settings.hits.hits[0]._source.kibana_settings.xpack.default_admin_email : null;ctx.vars.is_new = ctx.vars.fails_check && !ctx.vars.not_resolved;ctx.vars.is_resolve ...:347) ~[?:?]
        ... 12 more
[2018-10-16T11:35:48,371][ERROR][o.e.x.w.t.s.ExecutableScriptTransform] [OGAyjwj] failed to execute [script] transform for [ilnu0qQESvGSag2o4e3pJw_elasticsearch_cluster_status_6a91d937-af19-4b15-8613-33ca8d1e9f2d-2018-10-16T09:35:48.346Z]
org.elasticsearch.script.ScriptException: runtime error
        at org.elasticsearch.painless.PainlessScript.convertToScriptException(PainlessScript.java:101) ~[?:?]
        at org.elasticsearch.painless.PainlessScript$Script.execute(ctx.vars.email_recipient = (ctx.payload.kibana_settings.hits.total > 0) ? ctx.payload.kibana_settings.hits.hits[0]._source.kibana_settings.xpack.default_admin_email : null;ctx.vars.is_new = ctx.vars.fails_check && !ctx.vars.not_resolved;ctx.vars.is_resolve ...:1070) ~[?:?]
        at org.elasticsearch.painless.ScriptImpl.run(ScriptImpl.java:105) ~[?:?]
        at org.elasticsearch.xpack.watcher.transform.script.ExecutableScriptTransform.doExecute(ExecutableScriptTransform.java:69) ~[x-pack-watcher-6.2.3.jar:6.2.3]
        at org.elasticsearch.xpack.watcher.transform.script.ExecutableScriptTransform.execute(ExecutableScriptTransform.java:53) ~[x-pack-watcher-6.2.3.jar:6.2.3]
        at org.elasticsearch.xpack.watcher.transform.script.ExecutableScriptTransform.execute(ExecutableScriptTransform.java:38) ~[x-pack-watcher-6.2.3.jar:6.2.3]
        at org.elasticsearch.xpack.watcher.execution.ExecutionService.executeInner(ExecutionService.java:481) ~[x-pack-watcher-6.2.3.jar:6.2.3]
        at org.elasticsearch.xpack.watcher.execution.ExecutionService.execute(ExecutionService.java:322) ~[x-pack-watcher-6.2.3.jar:6.2.3]
        at org.elasticsearch.xpack.watcher.execution.ExecutionService.lambda$executeAsync$7(ExecutionService.java:426) ~[x-pack-watcher-6.2.3.jar:6.2.3]
        at org.elasticsearch.xpack.watcher.execution.ExecutionService$WatchExecutionTask.run(ExecutionService.java:580) [x-pack-watcher-6.2.3.jar:6.2.3]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:573) [elasticsearch-6.2.3.jar:6.2.3]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:1.8.0_111]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:1.8.0_111]
        at java.lang.Thread.run(Unknown Source) [?:1.8.0_111]
Caused by: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
        at java.util.ArrayList.rangeCheck(Unknown Source) ~[?:1.8.0_111]
        at java.util.ArrayList.get(Unknown Source) ~[?:1.8.0_111]
        at org.elasticsearch.painless.PainlessScript$Script.execute(ctx.vars.email_recipient = (ctx.payload.kibana_settings.hits.total > 0) ? ctx.payload.kibana_settings.hits.hits[0]._source.kibana_settings.xpack.default_admin_email : null;ctx.vars.is_new = ctx.vars.fails_check && !ctx.vars.not_resolved;ctx.vars.is_resolve ...:347) ~[?:?]
        ... 12 more

I know about the security license, but my license was expired some months ago. Any idea?

2

What makes you think that this is the culprit? The above is not coming from a query, but from a watch execution.

Also can you explain in more detail what works very slow means? Is indexing slow? Is searching slow?

Please take some more time to properly write up an issue, especially about the changed behaviour.

Thank you!

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.