ECK on IKS with fluentd. How to deal with secrets

Elastic Orchestration Elastic Cloud on Kubernetes (ECK)
1

Hello, I'm checking Quickstart | Elastic Cloud on Kubernetes [2.10] | Elastic and deployed elasticsearch + kibana on IKS with no major issues. Now I'm trying to deploy fluentd with helm chart

helm install stable/fluentd-elasticsearch

and obviously receive some security error. Please let me know how to deal with this

2019-08-19 14:13:17 +0000 [warn]: suppressed same stacktrace
2019-08-19 14:13:17 +0000 [warn]: [elasticsearch] failed to flush the buffer. retry_time=22 next_retry_seconds=2019-08-19 14:13:48 +0000 chunk="590753a76d953933a14e9c6883b6892e" error_class=Fluent::Plugin::ElasticsearchOutput::ConnectionFailure error="Can not reach Elasticsearch cluster ({:host=>"elasticsearch-client", :port=>9200, :scheme=>"http"})!"
2019-08-19 14:13:17 +0000 [warn]: suppressed same stacktrace
2019-08-19 14:13:32 +0000 [warn]: [kubelet.log] got incomplete line before first line from /var/log/kubelet.log:
"Aug 19 14:13:32 kube-bld65cjd0gighkrkfff0-mycluster-default-00000172 kubelet.service[24449]: W0819 14:13:32.675534
24449 kubelet_pods.go:832] Unable to retrieve pull secret kube-system/bluemix-default-secret for kube-system/ibm-keepalived-watcher-pcb7h due to secrets
"bluemix-default-secret" not found. The image pull may not succeed.\n"
2019-08-19 14:13:37 +0000 [warn]: [kubelet.log] got incomplete line before first line from /var/log/kubelet.log:
"Aug 19 14:13:37 kube-bld65cjd0gighkrkfff0-mycluster-default-00000172 kubelet.service[24449]: W0819 14:13:37.674962
24449 kubelet_pods.go:832] Unable to retrieve pull secret kube-system/bluemix-default-secret for kube-system/public-crbld65cjd0gighkrkfff0-alb1-566c8969f6-wrcwf due to secrets
"bluemix-default-secret" not found. The image pull may not succeed.\n"

2

Hello, it looks like that chart has been deprecated but the new repo documents how to add ES auth (the elasticsearch.auth config section):

3

@Anya_Sabo
Thank you, this helped

4

@Anya_Sabo

Do you have any idea how to fix this issue with fluentd on IBM IKS?

I need Dockerfile which is used for helmchart kiwigrid/helm-charts

Because these Dockerfiles are different https://github.com/fluent/fluentd-kubernetes-daemonset/tree/master/docker-image/v1.4

5

Hey @O_K, can you clarify what problem you're having? That said I'm not too familiar with the different fluentd charts, so if those are what you're having issues with it may be more helpful to go through the support routes those projects use.

6

On IBM IKS there is some mounting issue while using fluentd helm chart (I think I could fix it after playing with fluentd Dockerfile I found) for docker log volumes (/var/log/pods /var/log/containers), all in all, I started using Filebeat instead of fluentd and it works for me, at least I can see PODs logs in Kibana