Plugin installation on local ElasticSearch

Elastic Stack Elasticsearch
1

Hello, I was able to successfully install ElasticSearch 7.12.0 on my Mac and also work with it (mapping and indexing first objects via non local network). I want to install a plugin, but I can't get pass this error:

***-Mac-mini:elasticsearch-7.12.0 ***$ sudo bin/elasticsearch-plugin install https://github.com/mvSapphire/skiplist-elastic-plugin/blob/master/target/releases/searchFilter-plugin-7.2.0-SNAPSHOT.zip?raw=true 
Password: ******
-> Installing https://github.com/mvSapphire/skiplist-elastic-plugin/blob/master/target/releases/searchFilter-plugin-7.2.0-SNAPSHOT.zip?raw=true
-> Downloading https://github.com/mvSapphire/skiplist-elastic-plugin/blob/master/target/releases/searchFilter-plugin-7.2.0-SNAPSHOT.zip?raw=true
[=================================================] 100%
-> Failed installing https://github.com/mvSapphire/skiplist-elastic-plugin/blob/master/target/releases/searchFilter-plugin-7.2.0-SNAPSHOT.zip?raw=true
-> Rolling back https://github.com/mvSapphire/skiplist-elastic-plugin/blob/master/target/releases/searchFilter-plugin-7.2.0-SNAPSHOT.zip?raw=true
-> Rolled back https://github.com/mvSapphire/skiplist-elastic-plugin/blob/master/target/releases/searchFilter-plugin-7.2.0-SNAPSHOT.zip?raw=true
Exception in thread "main" java.nio.file.NoSuchFileException: /Users/lukassmilek/Downloads/elasticsearch-7.12.0/plugins/.installing-7959574728750973093/plugin-descriptor.properties
at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:218)
at java.base/java.nio.file.Files.newByteChannel(Files.java:375)
at java.base/java.nio.file.Files.newByteChannel(Files.java:426)
at java.base/java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:420)
at java.base/java.nio.file.Files.newInputStream(Files.java:160)
at org.elasticsearch.plugins.PluginInfo.readFromProperties(PluginInfo.java:173)
at org.elasticsearch.plugins.InstallPluginCommand.loadPluginInfo(InstallPluginCommand.java:786)
at org.elasticsearch.plugins.InstallPluginCommand.installPlugin(InstallPluginCommand.java:845)
at org.elasticsearch.plugins.InstallPluginCommand.execute(InstallPluginCommand.java:245)
at org.elasticsearch.plugins.InstallPluginCommand.execute(InstallPluginCommand.java:215)
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:75)
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:116)
at org.elasticsearch.cli.MultiCommand.execute(MultiCommand.java:80)
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:116)
at org.elasticsearch.cli.Command.main(Command.java:79)
at org.elasticsearch.plugins.PluginCli.main(PluginCli.java:36)

Is there anything else I have to install beside ElasticSearch? I tried few different plugins, but it behaves the same.

2

I don't really know but according to the documentation (GitHub - mvSapphire/skiplist-elastic-plugin: Plugin for elastic (7.2.0) to skip large lists of _ids) this plugin has not been updated and released for 7.2.0.

You should ask the author IMO.

1 Like
3

I believe it is only compiling of the plugin as the plugin itself is a short and simple. Is there any guide how to build your own plugin from scratch for users that do come from totally different industry and do not know much about programming?

What I found is:
medium article
elastic.co help, not very helpful
and some outdated posts...

I installed Visual Studio Code and tried to follow the Medium article above, but I hit following error on gradle assemble

Failed to apply plugin class 'org.elasticsearch.gradle.info.GlobalBuildInfoPlugin'.

Could not create plugin of type 'GlobalBuildInfoPlugin'.
Could not generate a decorated class for type GlobalBuildInfoPlugin.
org/gradle/jvm/toolchain/internal/SharedJavaInstallationRegistry

4

I'm afraid I can't help on that.

I just pinged the author of the updated version of the plugin in the issue you opened.

Could you also explain what you are trying to achieve with this plugin? So someone from the search team could may find a workaround or an existing way for doing it?

5

I know there is terms lookup function that can be used as a skip list in the ElasticSearch. This would require that the index gets updated very often to have an actual skip list available. The idea is basically what the above mentioned plugin does. original plugin

with help of RoaringBitmap I would like to pass an base64 string to the query script that would exclude documents that has a certain Integer value in a field XY

this will allow to pass the skip list effectively to query and have the actual state in the main MongoDB database always up to date

6

I was able to compile it by my own with Intelli Idea right now, followings the steps here

but it still throws the same during installation. Therefore I think it is nothing related to the plugin itself, but setup of the paths and elasticsearch. Perhaps you would have any idea?

I added to my .bash_profile following, but without any difference:

export ES_HOME=/Users/lukassmilek/Downloads/elasticsearch-7.12.0
export ES_JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk-16.0.1.jdk/Contents/Home
export PATH=$ES_HOME/bin:$ES_JAVA_HOME/bin:$PATH

is there anything else like permissions or something to setup? I installed elasticsearch by extracting the file in above mentioned folder.

7

may be @MarkVieira could help for the gradle part?

Why not using a simple bool query with a must_not clause which contains the integer you want to exclude from the results? Would that work for you?

8

Such must_not would have to check against array of 10000+ integers for some users. I think it would be also heavy traffic in that case.

I managed now to "somehow" compile it in a way that it got installed. I am not sure what I did though. I think that provided made a difference in the pom.xml file

   <dependency>
        <groupId>org.elasticsearch</groupId>
        <artifactId>elasticsearch</artifactId>
        <version>${elasticsearch.version}</version>
        <scope>provided</scope>
    </dependency>

I'll share the feedback if it works or not. :slight_smile:

9

I see.

Just to be sure we are not missing anything, could you share some typical documents you have today and what would be a request like with the plugin and with the must_not way?

10

Regarding the Gradle build error you see, that's a Gradle version incompatibility. Looks like you're using Gradle 7.0 but the version of the plugin you're using requires Gradle 6.8.

11

the simplest implementation would be as following:

{
    "query": {
        "function_score": {
	        "min_score" : 0.1,
            "query": {
               "match_all": {}
            },
            "script_score" : {
                "script" : {
                  "lang" : "skiplist",
                  "source" : "roaring",
                  "params" : {
                      "field" : "bf",
                      "data" : "OjAAAAEAAAAAAAAAEAAAAAoA"
                  }
                }
            }
        }
    }
}

the base64 string is RoaringBitmap with only one integer set ("10") and the min_score 0.1 exclude all documents that has this Integer 10 in the "bf" field.

It seems to work as intended when I tried it yesterday. I will try to perform some testing on thousands of dummy data today. I think the problem was combination of the .bash_profile setting with provided in dependency, but I am not 100% sure.

I ended up compiling it with Intelli J Idea as this was a bit simpler for a no-experience guy like me. But you must be correct, the VS Code added second Gradle as I probably missed versioning in the code.

Thank you, highly appreciated your help!

12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.