How to make sure that whitelisted plugin for painless is working?

gowthamkaruturi · August 23, 2021, 11:52am

Hi
I have created a sample whitelist plugin for painless, so that i can use the whitelisted methods in my painless script.
i used the following guide to have custom hexdigest methods compiled in to plugin

Aim of the plugin is to whitelist hexdigest methods so that painless scripts get access, so there is one such example i followed

i have created something like below

github.com

gowthamkaruturi/hexdigestplugin/blob/dev/src/main/java/com/gowtham/elasticsearch/hexdigestplugin/WhitelistPlugin.java

package com.gowtham.elasticsearch.hexdigestplugin;


import org.elasticsearch.plugins.Plugin;

public class WhitelistPlugin extends Plugin {
    // we don't actually need anything here, since whitelists are extended through SPI
}

github.com

gowthamkaruturi/hexdigestplugin/blob/dev/src/main/java/com/gowtham/elasticsearch/hexdigestplugin/WhitelistedExtension.java

package com.gowtham.elasticsearch.hexdigestplugin;

import org.elasticsearch.painless.spi.PainlessExtension;
import org.elasticsearch.painless.spi.Whitelist;
import org.elasticsearch.painless.spi.WhitelistLoader;
import org.elasticsearch.script.ScriptContext;
import org.elasticsearch.script.SearchScript;

import java.util.*;

public class WhitelistedExtension implements PainlessExtension {
    private static final String[] BASE_WHITELIST_FILES = new String[] {
            "com.gowtham.elasticsearch.hexdigestplugin.HexDigest.txt"
    };
    private static final Whitelist WHITELIST =
            WhitelistLoader.loadFromResourceFiles(WhitelistedExtension.class,BASE_WHITELIST_FILES);
    @Override
    public Map<ScriptContext<?>, List<Whitelist>> getContextWhitelists() {
//        HexDigest hdi = new HexDigest();
//        WhitelistInstanceBinding hexdigest = new WhitelistInstanceBinding("getHexDigest", hdi,"getHexDigest","String",Collections.singletonList("String"));

This file has been truncated. show original

i have installed plugin successfully and restarted the es successfully , logs displayed with the plugin i installed after i restarted.
But when i run search it gives me a exception.

curl -XGET -H "Content-Type: application/json"  http://localhost:9200/_search\?size\=10 -d '{"query":{"bool":{"filter":[{"range":{"meta_ts":{"gte":"2021-08-04T13:00:00.000Z","lte":"2021-08-23T13:51:35.698Z"}}},{"script":{"script":{"lang":"painless","source":"\n String convertedHexValue = null; def hd = new HexDigest(); String value = doc[params.field].value; String hex = (params.hex.length() % 2 == 0) ? params.hex : '0' + params.hex; if (value.length() > 0) { convertedHexValue = hd.getHexDigest(params.algorithm,value); } if (hex.equals(convertedHexValue)){ return true; } return false; \n ","params":{"field":"class","algorithm":"MD5","hex":"0449822057c99079dd9dc9818e300eaa"}}}}]}}}' | jq . | less

response form search

    "lang": "painless",
            "caused_by": {
              "type": "illegal_argument_exception",
              "reason": "invalid sequence of tokens near ['HexDigest'].",
              "caused_by": {
                "type": "no_viable_alt_exception",
                "reason": null

elastic search logs are

Caused by: org.antlr.v4.runtime.NoViableAltException
	at org.antlr.v4.runtime.atn.ParserATNSimulator.noViableAlt(ParserATNSimulator.java:1884) ~[?:?]
	at org.antlr.v4.runtime.atn.ParserATNSimulator.execATN(ParserATNSimulator.java:486) ~[?:?]
	at org.antlr.v4.runtime.atn.ParserATNSimulator.adaptivePredict(ParserATNSimulator.java:412) ~[?:?]
	at org.elasticsearch.painless.antlr.PainlessParser.unary(PainlessParser.java:2056) ~[?:?]
	at org.elasticsearch.painless.antlr.PainlessParser.expression(PainlessParser.java:1716) ~[?:?]
	at org.elasticsearch.painless.antlr.PainlessParser.declvar(PainlessParser.java:1466) ~[?:?]
	at org.elasticsearch.painless.antlr.PainlessParser.declaration(PainlessParser.java:1342) ~[?:?]
	at org.elasticsearch.painless.antlr.PainlessParser.dstatement(PainlessParser.java:964) ~[?:?]
	at org.elasticsearch.painless.antlr.PainlessParser.statement(PainlessParser.java:395) ~[?:?]
	at org.elasticsearch.painless.antlr.PainlessParser.source(PainlessParser.java:178) ~[?:?]
	at org.elasticsearch.painless.antlr.Walker.buildAntlrTree(Walker.java:222) ~[?:?]
	at org.elasticsearch.painless.antlr.Walker.<init>(Walker.java:204) ~[?:?]
	at org.elasticsearch.painless.antlr.Walker.buildPainlessTree(Walker.java:179) ~[?:?]
	at org.elasticsearch.painless.Compiler.compile(Compiler.java:211) ~[?:?]
	at org.elasticsearch.painless.PainlessScriptEngine$5.run(PainlessScriptEngine.java:420) ~[?:?]
	at org.elasticsearch.painless.PainlessScriptEngine$5.run(PainlessScriptEngine.java:416) ~[?:?]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_271]
	at org.elasticsearch.painless.PainlessScriptEngine.compile(PainlessScriptEngine.java:416) ~[?:?]
	at org.elasticsearch.painless.PainlessScriptEngine.compile(PainlessScriptEngine.java:167) ~[?:?]
	at org.elasticsearch.script.ScriptService.compile(ScriptService.java:363) ~[elasticsearch-6.8.18.jar:6.8.18]
	at org.elasticsearch.index.query.ScriptQueryBuilder.doToQuery(ScriptQueryBuilder.java:135) ~[elasticsearch-6.8.18.jar:6.8.18]
	at org.elasticsearch.index.query.AbstractQueryBuilder.toQuery(AbstractQueryBuilder.java:106) ~[elasticsearch-6.8.18.jar:6.8.18]
	at org.elasticsearch.index.query.AbstractQueryBuilder.toFilter(AbstractQueryBuilder.java:128) ~[elasticsearch-6.8.18.jar:6.8.18]
	at org.elasticsearch.index.query.BoolQueryBuilder.addBooleanClauses(BoolQueryBuilder.java:419) ~[elasticsearch-6.8.18.jar:6.8.18]
	at org.elasticsearch.index.query.BoolQueryBuilder.doToQuery(BoolQueryBuilder.java:386) ~[elasticsearch-6.8.18.jar:6.8.18]
	at org.elasticsearch.index.query.AbstractQueryBuilder.toQuery(AbstractQueryBuilder.java:106) ~[elasticsearch-6.8.18.jar:6.8.18]
	at org.elasticsearch.index.query.QueryShardContext.lambda$toQuery$2(QueryShardContext.java:319) ~[elasticsearch-6.8.18.jar:6.8.18]
	at org.elasticsearch.index.query.QueryShardContext.toQuery(QueryShardContext.java:331) ~[elasticsearch-6.8.18.jar:6.8.18]
	at org.elasticsearch.index.query.QueryShardContext.toQuery(QueryShardContext.java:318) ~[elasticsearch-6.8.18.jar:6.8.18]
	at org.elasticsearch.search.SearchService.parseSource(SearchService.java:806) ~[elasticsearch-6.8.18.jar:6.8.18]
	at org.elasticsearch.search.SearchService.createContext(SearchService.java:653) ~[elasticsearch-6.8.18.jar:6.8.18]
	at org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:604) ~[elasticsearch-6.8.18.jar:6.8.18]
	at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:388) ~[elasticsearch-6.8.18.jar:6.8.18]
	at org.elasticsearch.search.SearchService.access$100(SearchService.java:127) ~[elasticsearch-6.8.18.jar:6.8.18]
	at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:360) ~[elasticsearch-6.8.18.jar:6.8.18]
	at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:356) ~[elasticsearch-6.8.18.jar:6.8.18]
	at org.elasticsearch.search.SearchService$4.doRun(SearchService.java:1129) ~[elasticsearch-6.8.18.jar:6.8.18]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-6.8.18.jar:6.8.18]
	at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:41) ~[elasticsearch-6.8.18.jar:6.8.18]
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:778) ~[elasticsearch-6.8.18.jar:6.8.18]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-6.8.18.jar:6.8.18]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_271]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_271]
	at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_271]

can someone please let me know if the plugin is right or wrong?

Jack_Conradson · August 23, 2021, 6:11pm

Hi Gowtham!

This looks really close to me. I believe the issue is with the context the whitelist is applied to. For a filter query it actually uses a different context than search. If you instead change it to add this whitelist to FilterScript.CONTEXT in the WhitelistExtention class that should fix this issue.

As a side note the error you received indicates that the type wasn't found as part of that script. We have improved this error message in newer versions of Elasticsearch that would tell you symbol not found instead of a grammar exception with little information.

Please let me know if this fixes the issue. Thank you!

gowthamkaruturi · August 24, 2021, 2:52pm

thanks @Jack_Conradson
i had issues with plugin generation and i missed the FilterScript.context as u mentioned.
I have rectified both and it works as expected

system · September 21, 2021, 2:53pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
How do you use the ExampleWhitelistExtension in Terms Aggregations Elasticsearch	5	1251	March 22, 2018
Can't use example plugin in painless script Elasticsearch painless	1	494	April 15, 2021
PainlessExtension Custom Plugin is not loading Elasticsearch painless	1	508	March 17, 2019
Import own java library for painless Elasticsearch painless	2	777	November 28, 2019
Embbeed elastic search, how to install painless plugin Elasticsearch	3	1155	December 5, 2017

How to make sure that whitelisted plugin for painless is working?

Related topics