We use an OpenID Connect provider for Authentication & Authorisation in Elasticsearch, however have been tasked with locking down egress from our cluster (we're running in Kubernetes using Docker images).
We've implemented a proxy through which we'd like traffic to be directed and while this appears to have worked for our SLM connection to S3, we're unable to connect through Kibana to Elasticsearch using the OIDC realm because the connection to our OIDC Provider times out with log output similar to:
{"type": "server", "timestamp": "2020-03-10T20:23:35,364Z", "level": "WARN", "component": "o.e.x.s.a.AuthenticationService", "cluster.name": "my-elasticsearch", "node.name": "elasticsearch-1.elasticsearch-hs.my.svc.cluster.local", "message": "Authentication to realm kibana failed - Failed to authenticate user with OpenID Connect (Caused by ElasticsearchSecurityException[Failed to exchange code for Id Token using the Token Endpoint.]; nested: ConnectException[Timeout connecting to [my.oidc.provider.co.uk/1.2.3.4:443]];)", "cluster.uuid": "abcdef", "node.id": "12345" }
I can see from our proxy logs that while other components are successfully directing traffic through to the OIDC Provider, nothing ever appears from Elasticsearch, instead it appear to be trying to connect directly.
I've set the following (typical) environment variables:
- HTTP_PROXY (our proxy)
- HTTPS_PROXY (our proxy)
- NO_PROXY (localhost and intra-cluster services)
I also tried setting the lowercase variants of these same variables. Finally, I've tried adding the Java flags to the jvm.options file:
- -Dhttp.proxyHost=
- -Dhttp.proxyPort=
- -Dhttps.proxyHost=
- -Dhttps.proxyPort=
- -Dhttp.nonProxyHosts=
We're using Elasticsearch 7.6.0. Is it just that the OIDC classes fail to pay attention to proxy settings or are we missing something?