It appears that you have to do manual configuration of ES if you want to change items in the kibana.yml such as elasticsearch.requestHeadersWhitelist? is this correct and we cant use elasticsearchRef to do all the username and cert stuff for us?
Yes, you can combine both. We merge the settings you specify in the config section with the settings we derive from the elasticsearchRef
apiVersion: kibana.k8s.elastic.co/v1alpha1
kind: Kibana
metadata:
name: kibana-sample
spec:
version: 7.4.0
nodeCount: 1
config:
elasticsearch.requestHeadersWhitelist:
- authorization # or whatever you want here
elasticsearchRef:
name: "elasticsearch-sample"
In this example ECK takes care of the username, password, Elasticsearch URL and certificate authority setup but you can set a custom request header whitelist.
3 Likes
missed that it was config: and not elasticsearch:
Thanks!!
Hi,
I'm having a similar problem. I followed the guideline and created some custom configurations, but unfortunately they are not merged.
apiVersion: kibana.k8s.elastic.co/v1
kind: Kibana
metadata:
name: kibana
namespace: default
labels:
k8s-app: kibana
spec:
version: 7.5.2
count: 1
config:
xpack.security.enabled:
- true
xpack.security.audit.enabled:
- true
elasticsearch.username:
- elastic
elasticsearch.ssl.certificateAuthorities:
- /etc/certs/ca.crt
xpack.monitoring.kibana.collection.enabled:
- false
xpack.monitoring.ui.container.elasticsearch.enabled: false
elasticsearchRef:
name: "***"
...
When I have a look at the created config via kubectl I get different values:
[user@host]$ kubectl exec -it kibana -- cat config/kibana.yml
elasticsearch:
hosts:
- https://***:9200
password: ***
ssl:
certificateAuthorities: /etc/certs/ca.crt
verificationMode: certificate
username: elastic
server:
host: "0"
name: kibana
ssl:
certificate: /mnt/elastic-internal/http-certs/tls.crt
enabled: true
key: /mnt/elastic-internal/http-certs/tls.key
xpack:
monitoring:
ui:
container:
elasticsearch:
enabled: true
security:
audit:
enabled: true
enabled: true
Is this a known problem or did I do something wrong in my configuration file?
Greetings,
Samuel
Probably unrelated but these settings values should not be arrays, but single values. For example:
xpack.security.audit.enabled: true
As to why the values are not propagated correctly into the pod:
First, just to double check are we looking at the correct pod?
because I would expect the pod to be named something like kibana-kb-SOME_HASH
If it is the correct pod, we could double check if the reconciliation actually went through and updated the configuration. For that it would be worth checking the operator logs as described here:
Hi Peter,
Thanks for getting back to me. Yes the podname is corrent (I sanatized the -kb-SOME_HASH) to make it more readable. I also tried setting the values to single values as you suggested but there is no change in the behavior.
config:
xpack.security.enabled: true
xpack.security.audit.enabled: true
elasticsearch.username: elastic
elasticsearch.ssl.certificateAuthorities:
- /etc/certs/ca.crt
xpack.monitoring.kibana.collection.enabled: false
xpack.monitoring.ui.container.elasticsearch.enabled: false
Below you can find the operator logs (only errors) which show up when recreating the kibana instance. After this the error messages just keep repeating themselves. There might be a rather obvious reason("User-configured use is unsupported") , why I can not set the parameters. If this is the case, how am I supposed to set other values?
{"level":"error","@timestamp":"2020-02-05T13:34:26.968Z","logger":"controller runtime.controller","message":"Reconciler error","ver":"1.0.0-6881438d","controller":"kibana association-controller","request":"default/secmon-kibana","error":"Operation cannot be fulfilled on kibanas.kibana.k8s.elastic.co \"secmon-kibana\": the object has been modified; please apply your changes to the latest version and try again","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/zapr@v0.1.0/zapr.go:128\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.4.0/pkg/internal/controller/controller.go:258\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.4.0/pkg/internal/controller/controller.go:232\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.4.0/pkg/internal/controller/controller.go:211\nk8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/go/pkg/mod/k8s.io apimachinery@v0.0.0-20191028221656-72ed19daf4bb/pkg/util/wait/wait.go:152\nk8s.io apimachinery/pkg/util/wait.JitterUntil\n\t/go/pkg/mod/k8s.io/apimachinery@v0.0.0-20191028221656-72ed19daf4bb/pkg/util/wait/wait.go:153\nk8s.io/apimachinery/pkg/util/wait.Until\n\t/go/pkg/mod/k8s.io/apimachinery@v0.0.0-20191028221656-72ed19daf4bb/pkg/util/wait/wait.go:88"}
{"level":"info","@timestamp":"2020-02-05T13:34:28.054Z","logger":"elasticsearch-controller","message":"Elasticsearch manifest has warnings. Proceed at your own risk. [spec.nodeSets[0].config.xpack.security.enabled: Forbidden: Configuration setting is reserved for internal use. User-configured use is unsupported, spec.nodeSets[0].config.xpack.security.http.ssl.enabled: Forbidden: Configuration setting is reserved for internal use. User-configured use is unsupported, spec.nodeSets[0].config.xpack.security.transport.ssl.enabled: Forbidden: Configuration setting is reserved for internal use. User-configured use is unsupported, spec.nodeSets[0].config.xpack.security.transport.ssl.verification_mode: Forbidden: Configuration setting is reserved for internal use. User-configured use is unsupported, spec.nodeSets[1].config.xpack.security.enabled: Forbidden: Configuration setting is reserved for internal use. User-configured use is unsupported, spec.nodeSets[1].config.xpack.security.http.ssl.enabled: Forbidden: Configuration setting is reserved for internal use. User-configured use is unsupported, spec.nodeSets[1].config.xpack.security.transport.ssl.enabled: Forbidden: Configuration setting is reserved for internal use. User-configured use is unsupported, spec.nodeSets[1].config.xpack.security.transport.ssl.verification_mode: Forbidden: Configuration setting is reserved for internal use. User-configured use is unsupported]","ver":"1.0.0-6881438d","namespace":"default","es_name":"secmon"}
{"level":"error","@timestamp":"2020-02-05T13:34:29.999Z","logger":"controller-runtime.controller","message":"Reconciler error","ver":"1.0.0-6881438d","controller":"kibana-controller","request":"default/secmon-kibana","error":"Deployment.apps \"secmon-kibana-kb\" is invalid: spec.template.spec.containers[0].image: Required value","errorCauses":[{"error":"Deployment.apps \"secmon-kibana-kb\" is invalid: spec.template.spec.containers[0].image: Required value"}],"stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/zapr@v0.1.0/zapr.go:128\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.4.0/pkg/internal/controller/controller.go:258\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.4.0/pkg/internal/controller/controller.go:232\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.4.0/pkg/internal/controller/controller.go:211\nk8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/go/pkg/mod/k8s.io/apimachinery@v0.0.0-20191028221656-72ed19daf4bb/pkg/util/wait/wait.go:152\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/pkg/mod/k8s.io/apimachinery@v0.0.0-20191028221656-72ed19daf4bb/pkg/util/wait/wait.go:153\nk8s.io/apimachinery/pkg/util/wait.Until\n\t/go/pkg/mod/k8s.io/apimachinery@v0.0.0-20191028221656-72ed19daf4bb/pkg/util/wait/wait.go:88"}
Greetings
Samuel
These are just warnings and are referring to your Elasticsearch configuration, where you are apparently overriding values that we consider managed by ECK itself (see Settings managed by ECK | Elastic Cloud on Kubernetes [2.10] | Elastic)
If you want to set your own settings in the config section you can just do that, no need to include all the settings internally managed by ECK. ECK will merge your settings with the settings it generates internally, where your settings take precedence over anything set by ECK.
As to why you Kibana configuration does not line up with what you have in your manifest:
This seems relevant as it indicates that ECK was not able to update your Kibana deployment. So you are still seeing an old version of the deployment. Are you by any chance setting an empty Docker image or overriding the podTemplate for the Kibana deployment, in any way?
That might have been the problem. I destroyed every manually and now it seems to work fine. Thanks for helping out.