Trying to include tensorflow library in my ES plugin and getting the following error

[ashwin91kumar (Ashwin Kumar)]

#8 6.878 -> Rolled back file:///plugins/plugin.zip
#8 6.878 Exception in thread "main" java.lang.IllegalStateException: codebase property already set: codebase.tensorflow-core-api -> file:/usr/share/elasticsearch/plugins/.installing-7277063629313084784/tensorflow-core-api-0.4.0.jar, cannot set to file:/usr/share/elasticsearch/plugins/.installing-7277063629313084784/tensorflow-core-api-0.4.0-linux-x86_64.jar
#8 6.879 	at org.elasticsearch.bootstrap.PolicyUtil.readPolicy(PolicyUtil.java:268)
#8 6.879 	at org.elasticsearch.bootstrap.PolicyUtil.readPolicyInfo(PolicyUtil.java:333)
#8 6.879 	at org.elasticsearch.bootstrap.PolicyUtil.getPluginPolicyInfo(PolicyUtil.java:370)
#8 6.879 	at org.elasticsearch.plugins.cli.InstallPluginAction.installPlugin(InstallPluginAction.java:889)
#8 6.879 	at org.elasticsearch.plugins.cli.InstallPluginAction.execute(InstallPluginAction.java:245)
#8 6.879 	at org.elasticsearch.plugins.cli.InstallPluginCommand.execute(InstallPluginCommand.java:88)
#8 6.879 	at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:77)
#8 6.879 	at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:112)
#8 6.879 	at org.elasticsearch.cli.MultiCommand.execute(MultiCommand.java:95)
#8 6.879 	at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:112)
#8 6.879 	at org.elasticsearch.cli.Command.main(Command.java:77)
#8 6.879 	at org.elasticsearch.plugins.cli.PluginCli.main(PluginCli.java:36)
#8 ERROR: executor failed running [/bin/sh -c elasticsearch-plugin install --batch file:///plugins/selection-es-rescore-plugin.zip]: exit code: 1

I'm trying to add the following dependency for my plugin:

 implementation( group: 'org.tensorflow', name: 'tensorflow-core-platform', version: '0.4.0')

I am getting the above exception. Can anyone help to resolve this?

[spinscale (Alexander Reelsen)]

Can you share the policy file of that plugin along with the output of unzip -l of the packaged plugin?

[ashwin91kumar (Ashwin Kumar)]

Hi @spinscale ,
This is the contents of policy file:

grant {
    permission java.io.FilePermission "/app/models/saved_model/*", "read";
    permission java.lang.RuntimePermission "getClassLoader";
    permission java.lang.RuntimePermission "setContextClassLoader";
    permission java.lang.RuntimePermission "accessDeclaredMembers";
    permission java.util.PropertyPermission "*", "read, write";
};

This is the output of the unzip -l:

  Length      Date    Time    Name
---------  ---------- -----   ----
     1705  03-25-2022 15:07   plugin-descriptor.properties
    57096  03-25-2022 15:07  es-rescore-plugin.jar
     2371  03-25-2022 10:23   tensorflow-core-platform-0.4.0.jar
  8039455  03-25-2022 10:23   tensorflow-core-api-0.4.0.jar
101410625  03-25-2022 10:23   tensorflow-core-api-0.4.0-linux-x86_64.jar
101594815  03-25-2022 10:23   tensorflow-core-api-0.4.0-macosx-x86_64.jar
 58567796  03-25-2022 10:23   tensorflow-core-api-0.4.0-windows-x86_64.jar
   487566  03-25-2022 10:23   javacpp-1.5.6.jar
    25611  03-25-2022 10:23   javacpp-1.5.6-linux-x86_64.jar
    21901  03-25-2022 10:23   javacpp-1.5.6-macosx-x86_64.jar
  1305252  03-25-2022 10:23   javacpp-1.5.6-windows-x86_64.jar
  1635823  03-25-2022 10:23   protobuf-java-3.9.2.jar
   229133  03-15-2022 10:23   ndarray-0.3.3.jar
      503  03-25-2022 14:58   plugin-security.policy
---------                     -------
273379652                     14 files

TIA

[spinscale (Alexander Reelsen)]

Hey,

so I think there might be an issue with the platform specific jars, so that tensorflow-core-api-0.4.0.jar and tensorflow-core-api-0.4.0-linux-x86_64.jar are both summarized as tensorflow-core-api and thus the plugin installer thinks this is a duplicate.

Can you open an issue for this in Issues · elastic/elasticsearch · GitHub

Thanks!