Installation of plugins using initContainers stuck

Elastic Orchestration Elastic Cloud on Kubernetes (ECK)
1

Hi,

I was trying to install the repository-s3 plugin on my Kubernetes cluster using initContainers. However, the pod seems to get stuck when trying to install the plugin. So I removed the initContainer and went inside the pod when it started, and tried to install it manually, just for debugging purposes. But it had the same result.

I'm using ECK version 1.1.2 and this is my Elastic yaml file:

apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: project
spec:
  version: 7.8.0
  nodeSets:
  - name: default
    count: 1
    config:
      discovery.type: single-node

      node.master: true
      node.data: true
      node.ingest: true
    podTemplate:
      spec:
        initContainers:
        - name: sysctl
          securityContext:
            privileged: true
          command: ['sh', '-c', 'sysctl -w vm.max_map_count=262144']
        - name: install-plugins # For backups
          command: ['sh', '-c', '/usr/share/elasticsearch/bin/elasticsearch-plugin install repository-s3']

This is the output from the logs of the initContainer:

-> Installing repository-s3
-> Downloading repository-s3 from elastic
[=================================================] 100%??

It stays stuck there, and doesn't exit.

Anyone that has similar problems?

Thanks in advance!

2

It is hard to tell from the data you have shared what is going wrong with the plugin installation. Did you manage to solve the issue? I assume you have verified that network access works and the output from the init container does indicate that it had made some progress.

Because installing plugins via init containers has downsides like the one you have experienced (there are others like the fact that we have to repeat the process on every Pod and everytime we recreate a Pod due to configuration changes) we generally recommend to use custom Docker images for production use. We have documented this approach here: https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-custom-images.html

3

Hi Peter,

Thanks for your response.
I've deleted the ECK cluster, because I had to change some other things. After trying it again, but adding the --batch option, it seemed to work. I have no idea why and if it's related to the new cluster or to the --batch option. But it works!

And yeah, I've read about the custom images. I'll take a look at that. The only main advantage about the initContainers is that everything is done in the same yaml file instead of creating custom images and pulling them. But I'll definitely look into them!

4

It’s the batch parameter. I missed that in your original post. I believe the plugin installer is otherwise waiting for user input