Error from server (NotFound): services "quickstart-es-http" not found

So it looks like the operator is starting correctly (or at least started correctly once), because the webhook is getting created. But the API server cannot communicate with the webhook. In managed systems where the API server is external to the cluster, or if there are network policies in place that block inter-namespace communication this is expected. For managed vendors they will normally have documentation on how to allow communication.

That said, the current webhook configuration in 0.9.0 should be failing open, and indeed it looks like it is:


W0821 23:05:12.556120       1 dispatcher.go:67] Failed calling webhook, failing open validation.elasticsearch.elastic.co: failed calling webhook "validation.elasticsearch.elastic.co": Post https://elastic-webhook-service.elastic-system.svc:443/validate-elasticsearches?timeout=30s: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
E0821 23:05:12.556189       1 dispatcher.go:68] failed calling webhook "validation.elasticsearch.elastic.co": Post https://elastic-webhook-service.elastic-system.svc:443/validate-elasticsearches?timeout=30s: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

But it looks like a timeout might still be canceling the request for some reason. I see there are some upstream issues with a similar issue:

But I haven't been able to reproduce it. If you cannot resolve the communication issue between the API server and the operator then it may be worth it to remove the webhook (kubectl delete validatingwebhookconfigurations.admissionregistration.k8s.io validating-webhook-configuration), and configure the operator to not create the webhook automatically (see here).