6.0.0-beta2 Reindex from remote getProxySelector AccessControlException

[loren (Loren Siebert)]

I'm not sure why the JSM is complaining when I try to reindex to a local 6.0.0-beta2 instance from a remote cluster that I access via an SSH tunnel on another localhost port. I have this in my elasticsearch.yaml file:
reindex.remote.whitelist: localhost:9217

Then I run:

curl -XPOST "http://localhost:9262/_reindex" -H 'Content-Type: application/json' -d'
{
  "source": {
    "remote": {
      "host": "http://localhost:9217"
    },
    "index": "local_index",
    "query": {
      "match": {
        "content": "stuff"
      }
    }
  },
  "dest": {
    "index": "remote_index"
  }
}'

and I get

{"error":{"root_cause":[{"type":"access_control_exception","reason":"access denied (\"java.net.NetPermission\" \"getProxySelector\")"}],"type":"access_control_exception","reason":"access denied (\"java.net.NetPermission\" \"getProxySelector\")"},"status":500}

with this stacktrace

[2017-09-08T07:13:35,955][WARN ][r.suppressed             ] path: /_reindex, params: {}
java.security.AccessControlException: access denied ("java.net.NetPermission" "getProxySelector")
	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) ~[?:1.8.0_112]
	at java.security.AccessController.checkPermission(AccessController.java:884) ~[?:1.8.0_112]
	at java.lang.SecurityManager.checkPermission(SecurityManager.java:549) ~[?:1.8.0_112]
	at java.net.ProxySelector.getDefault(ProxySelector.java:94) ~[?:1.8.0_112]
	at org.apache.http.impl.nio.client.HttpAsyncClientBuilder.build(HttpAsyncClientBuilder.java:793) ~[?:?]
	at org.elasticsearch.client.RestClientBuilder$2.run(RestClientBuilder.java:214) ~[?:?]
	at org.elasticsearch.client.RestClientBuilder$2.run(RestClientBuilder.java:211) ~[?:?]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_112]
	at org.elasticsearch.client.RestClientBuilder.createHttpClient(RestClientBuilder.java:211) ~[?:?]
	at org.elasticsearch.client.RestClientBuilder.access$000(RestClientBuilder.java:40) ~[?:?]
	at org.elasticsearch.client.RestClientBuilder$1.run(RestClientBuilder.java:185) ~[?:?]
	at org.elasticsearch.client.RestClientBuilder$1.run(RestClientBuilder.java:182) ~[?:?]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_112]
	at org.elasticsearch.client.RestClientBuilder.build(RestClientBuilder.java:182) ~[?:?]
	at org.elasticsearch.index.reindex.TransportReindexAction.buildRestClient(TransportReindexAction.java:229) ~[?:?]

All the JSM information I see relates to plugins (I only have analysis-icu installed). Do I need to create a custom policy to reindex from remote?

[loren (Loren Siebert)]

I changed the grant in modules/reindex/plugin-security.policy from

grant codeBase "${codebase.elasticsearch-rest-client-6.0.0-beta2-SNAPSHOT.jar}" {
  // rest client uses system properties which gets the default proxy
  permission java.net.NetPermission "getProxySelector";
};

to

grant {
  // rest client uses system properties which gets the default proxy
  permission java.net.NetPermission "getProxySelector";
};

and _reindex works now.

That JAR file name seems correct to me, but maybe it's not. ¯_(ツ)_/¯

[jasontedor]

It's a bug in our build process. We discovered this internally, fixed it for forthcoming releases, and added a CI job so something like this never happens again. Sorry for the troubles.

Also, you can get a stricter grant by simply removing -SNAPSHOT from the original grant so they only the REST client JAR in the reindex classloader is granted the permission.

[jasontedor]

@loren I've added you to our Pioneer Program