Filter 'is between' on scripted fields throws error

Elastic Stack Kibana
1

Hi,

'is between' filter on scripted fields is giving shard failures.

I see the following error in elasticsearch logs:

org.elasticsearch.transport.RemoteTransportException: [elk-node][172.17.0.2:9300][indices:data/read/search[phase/query]]
Caused by: org.elasticsearch.search.query.QueryPhaseExecutionException: Query Failed [Failed to execute main query]
        at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:305) ~[elasticsearch-7.3.1.jar:7.3.1]
        at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:113) ~[elasticsearch-7.3.1.jar:7.3.1]
        at org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:335) ~[elasticsearch-7.3.1.jar:7.3.1]
        at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:360) ~[elasticsearch-7.3.1.jar:7.3.1]
        at org.elasticsearch.search.SearchService.lambda$executeQueryPhase$1(SearchService.java:340) ~[elasticsearch-7.3.1.jar:7.3.1]
        at org.elasticsearch.action.ActionListener.lambda$map$2(ActionListener.java:145) ~[elasticsearch-7.3.1.jar:7.3.1]
        at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:62) ~[elasticsearch-7.3.1.jar:7.3.1]
        at org.elasticsearch.search.SearchService$2.doRun(SearchService.java:1052) ~[elasticsearch-7.3.1.jar:7.3.1]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.3.1.jar:7.3.1]
        at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:44) ~[elasticsearch-7.3.1.jar:7.3.1]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:758) ~[elasticsearch-7.3.1.jar:7.3.1]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.3.1.jar:7.3.1]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
        at java.lang.Thread.run(Thread.java:835) [?:?]
Caused by: org.elasticsearch.script.ScriptException: runtime error
        at org.elasticsearch.painless.PainlessScript.convertToScriptException(PainlessScript.java:94) ~[?:?]
        at org.elasticsearch.painless.PainlessScript$Script.execute(boolean gte(Supplier s, def v) {return s.get() >= v} boolean lt(Supplier s, def v) {return s.get() < v}gte(() -> { if (!doc['some_field'].empty) { ...:434) ~[?:?]
        at org.elasticsearch.index.query.ScriptQueryBuilder$ScriptQuery$1$1.matches(ScriptQueryBuilder.java:185) ~[elasticsearch-7.3.1.jar:7.3.1]
        at org.apache.lucene.search.ConjunctionDISI$ConjunctionTwoPhaseIterator.matches(ConjunctionDISI.java:345) ~[lucene-core-8.1.0.jar:8.1.0 dbe5ed0b2f17677ca6c904ebae919363f2d36a0a - ishan - 2019-05-09 19:34:03]
        at org.apache.lucene.search.Weight$DefaultBulkScorer.scoreRange(Weight.java:250) ~[lucene-core-8.1.0.jar:8.1.0 dbe5ed0b2f17677ca6c904ebae919363f2d36a0a - ishan - 2019-05-09 19:34:03]
        at org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:229) ~[lucene-core-8.1.0.jar:8.1.0 dbe5ed0b2f17677ca6c904ebae919363f2d36a0a - ishan - 2019-05-09 19:34:03]
        at org.elasticsearch.search.internal.CancellableBulkScorer.score(CancellableBulkScorer.java:56) ~[elasticsearch-7.3.1.jar:7.3.1]
        at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:39) ~[lucene-core-8.1.0.jar:8.1.0 dbe5ed0b2f17677ca6c904ebae919363f2d36a0a - ishan - 2019-05-09 19:34:03]
        at org.elasticsearch.search.internal.ContextIndexSearcher.searchInternal(ContextIndexSearcher.java:198) ~[elasticsearch-7.3.1.jar:7.3.1]
        at org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:179) ~[elasticsearch-7.3.1.jar:7.3.1]
        at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:443) ~[lucene-core-8.1.0.jar:8.1.0 dbe5ed0b2f17677ca6c904ebae919363f2d36a0a - ishan - 2019-05-09 19:34:03]
        at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:270) ~[elasticsearch-7.3.1.jar:7.3.1]
        at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:113) ~[elasticsearch-7.3.1.jar:7.3.1]
        at org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:335) ~[elasticsearch-7.3.1.jar:7.3.1]
        at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:360) ~[elasticsearch-7.3.1.jar:7.3.1]
        at org.elasticsearch.search.SearchService.lambda$executeQueryPhase$1(SearchService.java:340) ~[elasticsearch-7.3.1.jar:7.3.1]
        at org.elasticsearch.action.ActionListener.lambda$map$2(ActionListener.java:145) ~[elasticsearch-7.3.1.jar:7.3.1]
        at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:62) ~[elasticsearch-7.3.1.jar:7.3.1]
        at org.elasticsearch.search.SearchService$2.doRun(SearchService.java:1052) ~[elasticsearch-7.3.1.jar:7.3.1]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.3.1.jar:7.3.1]
        at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:44) ~[elasticsearch-7.3.1.jar:7.3.1]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:758) ~[elasticsearch-7.3.1.jar:7.3.1]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.3.1.jar:7.3.1]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
        at java.lang.Thread.run(Thread.java:835) ~[?:?]
Caused by: java.lang.NullPointerException
        at org.elasticsearch.painless.DefBootstrap$PIC.fallback(DefBootstrap.java:207) ~[?:?]
        at org.elasticsearch.painless.PainlessScript$Script.lambda$0(boolean gte(Supplier s, def v) {return s.get() >= v} boolean lt(Supplier s, def v) {return s.get() < v}gte(() -> { if (!doc['some_field'].empty) { ...:189) ~[?:?]
        at org.elasticsearch.painless.PainlessScript$Script$$Lambda0.get(Unknown Source) ~[?:?]
        at org.elasticsearch.painless.PainlessScript$Script.gte(boolean gte(Supplier s, def v) {return s.get() >= v} boolean lt(Supplier s, def v) {return s.get() < v}gte(() -> { if (!doc['some_field'].empty) { ...:41) ~[?:?]
        at org.elasticsearch.painless.PainlessScript$Script.execute(boolean gte(Supplier s, def v) {return s.get() >= v} boolean lt(Supplier s, def v) {return s.get() < v}gte(() -> { if (!doc['some_field'].empty) { ...:265) ~[?:?]
        at org.elasticsearch.index.query.ScriptQueryBuilder$ScriptQuery$1$1.matches(ScriptQueryBuilder.java:185) ~[elasticsearch-7.3.1.jar:7.3.1]
        at org.apache.lucene.search.ConjunctionDISI$ConjunctionTwoPhaseIterator.matches(ConjunctionDISI.java:345) ~[lucene-core-8.1.0.jar:8.1.0 dbe5ed0b2f17677ca6c904ebae919363f2d36a0a - ishan - 2019-05-09 19:34:03]
        at org.apache.lucene.search.Weight$DefaultBulkScorer.scoreRange(Weight.java:250) ~[lucene-core-8.1.0.jar:8.1.0 dbe5ed0b2f17677ca6c904ebae919363f2d36a0a - ishan - 2019-05-09 19:34:03]
        at org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:229) ~[lucene-core-8.1.0.jar:8.1.0 dbe5ed0b2f17677ca6c904ebae919363f2d36a0a - ishan - 2019-05-09 19:34:03]
        at org.elasticsearch.search.internal.CancellableBulkScorer.score(CancellableBulkScorer.java:56) ~[elasticsearch-7.3.1.jar:7.3.1]
        at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:39) ~[lucene-core-8.1.0.jar:8.1.0 dbe5ed0b2f17677ca6c904ebae919363f2d36a0a - ishan - 2019-05-09 19:34:03]
        at org.elasticsearch.search.internal.ContextIndexSearcher.searchInternal(ContextIndexSearcher.java:198) ~[elasticsearch-7.3.1.jar:7.3.1]
        at org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:179) ~[elasticsearch-7.3.1.jar:7.3.1]
        at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:443) ~[lucene-core-8.1.0.jar:8.1.0 dbe5ed0b2f17677ca6c904ebae919363f2d36a0a - ishan - 2019-05-09 19:34:03]
        at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:270) ~[elasticsearch-7.3.1.jar:7.3.1]
        at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:113) ~[elasticsearch-7.3.1.jar:7.3.1]
        at org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:335) ~[elasticsearch-7.3.1.jar:7.3.1]
        at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:360) ~[elasticsearch-7.3.1.jar:7.3.1]
        at org.elasticsearch.search.SearchService.lambda$executeQueryPhase$1(SearchService.java:340) ~[elasticsearch-7.3.1.jar:7.3.1]
        at org.elasticsearch.action.ActionListener.lambda$map$2(ActionListener.java:145) ~[elasticsearch-7.3.1.jar:7.3.1]
        at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:62) ~[elasticsearch-7.3.1.jar:7.3.1]
        at org.elasticsearch.search.SearchService$2.doRun(SearchService.java:1052) ~[elasticsearch-7.3.1.jar:7.3.1]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.3.1.jar:7.3.1]
        at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:44) ~[elasticsearch-7.3.1.jar:7.3.1]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:758) ~[elasticsearch-7.3.1.jar:7.3.1]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.3.1.jar:7.3.1]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
        at java.lang.Thread.run(Thread.java:835) ~[?:?]
2

What version of the Elastic stack are you using?
What does your index mapping look like?
Are your index mappings consistent for all indices identified by your Kibana Index Pattern?

Can you provide a screen shot of the filter UI?

3

@Nathan_Reese thanks for the reply.

We are using ELK stack 7.3.1 basic license.

We have configured dynamic mapping for our indices except for some fields. Mappings are consistent for all indices of the pattern.

I couldn't upload screenshot but I get error 'x of y shards failed' when I use 'is between' filter on scripted field.

Let me know if you need more info.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.