amitsa
(AMIT KUMAR)
May 6, 2022, 6:52am
1
Hi i am trying to generate certificate and set passwd in interactive mode during build time but getting error.
Docker file
ARG ELASTICSEARCH_VER="8.1.3"
ARG ELASTICSEARCH_IMAGE=docker.elastic.co/elasticsearch/elasticsearch:${ELASTICSEARCH_VER}@sha256:577b382dda5d05385aea8c7b60dad97e02ff41ca0da54f723151c2aed9ac8f54
FROM ${ELASTICSEARCH_IMAGE}
COPY --chown=root:elasticsearch config/elasticsearch.yml /usr/share/elasticsearch/config/
COPY --chown=root:elasticsearch certificates/create-cert.sh /usr/share/elasticsearch/
RUN chmod +x /usr/share/elasticsearch/create-cert.sh \
&& /usr/share/elasticsearch/create-cert.sh
create-cert.sh
printf '\n\n'|bin/elasticsearch-certutil ca
printf '\n\n\n'|bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12
mv elastic-certificates.p12 config/
printf 'Y\nelastic\nelastic\napmsystem\napmsystem\nkibana\nkibana\nlogstash\nlogstash\nbeatssystem\nbeatssystem\nremote\nremote\n'|bin/elasticsearch-setup-passwords interactive
cd config/
openssl pkcs12 -passin pass:"" -in elastic-certificates.p12 -out cacert.pem --cacerts -nokeys
openssl pkcs12 -passin pass:"" -in elastic-certificates.p12 -out clients.crt.pem -clcerts -nokeys
openssl pkcs12 -passin pass:"" -in elastic-certificates.p12 -out clients.key.pem -nocerts -nodes
but getting below error during build time
=> ERROR [7/7] RUN bin/elasticsearch-setup-passwords interactive 2.4s
[7/7] RUN bin/elasticsearch-setup-passwords interactive:
#11 1.778
#11 1.778 ERROR: Elasticsearch keystore file is missing [/usr/share/elasticsearch/config/elasticsearch.keystore]
executor failed running [/bin/sh -c bin/elasticsearch-setup-passwords interactive]: exit code: 78
kindly help .
isn't keystore and elasticsearch.yml present in the image
Hi,
it will be great to share the Elasticsearch.yml file you're using in dockerfile.
ikakavas
(Ioannis Kakavas)
May 7, 2022, 8:15pm
3
You don’t have to set a password and generate certificates yourself, this happens automatically for you - Elasticsearch takes care of everything.
See the documentation at Install Elasticsearch with Docker | Elasticsearch Guide [8.2] | Elastic
amitsa
(AMIT KUMAR)
May 8, 2022, 1:12am
4
Its a multi node cluster deployment. i have docker image of Elasticsearch and a esrally client. I want to connect esrally to secure Elasticsearch. I am running on backend kubernetes.
Data Nodes
---
apiVersion: v1
kind: ConfigMap
metadata:
name: es-data-config
labels:
app: elasticsearch
role: data
data:
elasticsearch.yml: |-
cluster.name: ${CLUSTER_NAME}
node.name: ${NODE_NAME}
discovery.seed_hosts: ${NODE_LIST}
cluster.initial_master_nodes: ${MASTER_NODES}
network.host: 0.0.0.0
node.roles: data
xpack.security.enabled: true
xpack.monitoring.collection.enabled: false
ingest.geoip.downloader.enabled: false
#bootstrap.memory_lock: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
#xpack.security.http.ssl.enabled: true
#xpack.security.http.ssl.verification_mode: certificate
#xpack.security.http.ssl.keystore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
#xpack.security.http.ssl.truststore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
#xpack.security.http.ssl.client_authentication: optional
#xpack.security.http.ssl.certificate_authorities: /usr/share/elasticsearch/config/certs/cacert.pem
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: elasticsearch
labels:
app: elasticsearch
role: data
spec:
serviceName: es-data
replicas: 3
selector:
matchLabels:
app: elasticsearch
role: data
template:
metadata:
labels:
app: elasticsearch
role: data
spec:
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: role
operator: In
values:
- es-master
- esrally
- data
topologyKey: "kubernetes.io/hostname"
securityContext:
fsGroup: 1000
initContainers:
- name: sysctl
image: busybox
imagePullPolicy: IfNotPresent
command: ["sysctl", "-w", "vm.max_map_count=262144"]
securityContext:
privileged: true
# Increase the max number of open file descriptors.
- name: increase-fd-ulimit
image: busybox
imagePullPolicy: IfNotPresent
command: ["sh", "-c", "ulimit -n 65536"]
securityContext:
privileged: true
containers:
- name: elasticsearch
image: IMAGENAME(Dockerfile.1.elasticseach)
imagePullPolicy: Always
resources:
limits:
cpu: 16
memory: "34Gi"
ports:
- containerPort: 9300
name: transport
env:
- name: CLUSTER_NAME
value: elasticsearch
- name: NODE_NAME
value: es-data
- name: NODE_LIST
value: es-master,es-data
- name: MASTER_NODES
value: es-master
- name: "ES_JAVA_OPTS"
value: "-Xms31g -Xmx31g"
volumeMounts:
- name: config
mountPath: /usr/share/elasticsearch/config/elasticsearch.yml
readOnly: true
subPath: elasticsearch.yml
- name: es-data
mountPath: /dev/shm
volumes:
- name: config
configMap:
name: es-data-config
- name: es-data
emptyDir: {}
---
apiVersion: v1
kind: Service
metadata:
name: es-data
labels:
app: elasticsearch
role: data
spec:
selector:
app: elasticsearch
role: data
#clusterIP: None
ports:
- name: transport
port: 9300
Master Node
apiVersion: v1
kind: ConfigMap
metadata:
name: es-master-config
labels:
app: elasticsearch
role: es-master
data:
elasticsearch.yml: |-
cluster.name: ${CLUSTER_NAME}
node.name: ${NODE_NAME}
discovery.seed_hosts: ${NODE_LIST}
cluster.initial_master_nodes: ${MASTER_NODES}
network.host: 0.0.0.0
node.roles: master
xpack.security.enabled: true
xpack.monitoring.collection.enabled: false
ingest.geoip.downloader.enabled: false
#bootstrap.memory_lock: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.verification_mode: certificate
xpack.security.http.ssl.keystore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
xpack.security.http.ssl.truststore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
xpack.security.http.ssl.client_authentication: optional
#xpack.security.http.ssl.certificate_authorities: /usr/share/elasticsearch/config/certs/cacert.pem
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: elasticsearch
labels:
app: elasticsearch
role: es-master
spec:
replicas: 1
selector:
matchLabels:
app: elasticsearch
role: es-master
template:
metadata:
labels:
app: elasticsearch
role: es-master
spec:
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: role
operator: In
values:
- data
- esrally
topologyKey: "kubernetes.io/hostname"
podAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: role
operator: In
values:
- es-master
topologyKey: "kubernetes.io/hostname"
securityContext:
fsGroup: 1000
initContainers:
- name: sysctl
image: busybox
imagePullPolicy: IfNotPresent
command: ["sysctl", "-w", "vm.max_map_count=262144"]
securityContext:
privileged: true
# Increase the max number of open file descriptors.
- name: increase-fd-ulimit
image: busybox
imagePullPolicy: IfNotPresent
command: ["sh", "-c", "ulimit -n 65536"]
securityContext:
privileged: true
containers:
- name: elasticsearch
image: ELASTIC IMAGE(FROM DOCKER FILE)
imagePullPolicy: Always
resources:
limits:
cpu: 16
memory: "34Gi"
ports:
- containerPort: 9200
name: http
- containerPort: 9300
name: transport
env:
- name: CLUSTER_NAME
value: elasticsearch
- name: NODE_NAME
value: es-master
- name: NODE_LIST
value: es-master,es-data
- name: MASTER_NODES
value: es-master
- name: "ES_JAVA_OPTS"
value: "-Xms31g -Xmx31g"
volumeMounts:
- name: config
mountPath: /usr/share/elasticsearch/config/elasticsearch.yml
readOnly: true
subPath: elasticsearch.yml
- name: storage-master
mountPath: /dev/shm
volumes:
- name: config
configMap:
name: es-master-config
- name: storage-master
emptyDir:
medium: ""
---
apiVersion: v1
kind: Service
metadata:
name: es-master
labels:
service: elasticsearch
role: es-master
spec:
selector:
app: elasticsearch
role: es-master
#clusterIP: None
ports:
- name: http
port: 9200
- name: transport
port: 9300
ESRAlly NODE
apiVersion: batch/v1
kind: Job
metadata:
name: benchmark
labels:
role: esrally
spec:
template:
metadata:
name: benchmark
labels:
role: esrally
spec:
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: role
operator: In
values:
- data
- es-master
topologyKey: "kubernetes.io/hostname"
securityContext:
runAsUser: 1000
runAsGroup: 1000
fsGroup: 1000
initContainers:
- name: wait-for-es-service
image: curlimages/curl:latest
imagePullPolicy: IMAGEPOLICY(Always)
command: ["/bin/sh","-c","while [ $(curl --cacert //cacert.pem -k -sw '%{https_code}' -m 5 'https://es-master:9200' -o /dev/null) -ne 200 ];do echo Waiting...;sleep 1s;done"]
containers:
- name: benchmark
image: ESRALLY IMAGE
imagePullPolicy: Always
env:
- name: ELASTICSEARCH_URL
value: "http://es-master:9200"
- name: RALLY_RACE
value: "RALLY_RACE"
- name: RALLY_RACE_params_json
value: "RALLY_RACE_params_json"
- name: CHALLENGE
value: "CHALLENGE"
- name: TESTMODE
value: "TESTMODE"
volumeMounts:
- name: esrally-data
mountPath: /dev/shm
volumes:
- name: esrally-data
emptyDir: {}
restartPolicy: Never
RALLY COMMAND
ELASTIC_EP=https://es-master:9200
CLIENT_OPTIONS="basic_auth_user:elastic,basic_auth_password:DEFAULT PASSWD ,timeout:120,use_ssl:false,verify_certs:false,ca_certs:/rally/cacert.pem"
esrally race --offline --track-params=/rally/$RALLY_RACE_params_json --track-path=/rally/.rally/benchmarks/tracks/default/${RALLY_RACE} --pipeline=benchmark-only --target-hosts=${ELASTIC_EP} ${TESTMODE} --client-options ${CLIENT_OPTIONS} --report-format=csv
amitsa
(AMIT KUMAR)
May 8, 2022, 1:13am
5
Elasticsearch.yml is default one...i have removed that line. The deployment is on backend(kubernetes). So i want to generate certificate at build time and set the password as well during build time
amitsa
(AMIT KUMAR)
May 8, 2022, 1:15am
6
All the nodes are deployed on different host machine.
esrally ====> es-master ===> 3 es-data nodes(all on different host machine)
system
(system)
Closed
June 5, 2022, 1:15am
7
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.