I would like to have a single node ES (no Kibana involved) authenticate against Keycloak for all requests.
I've followed the OpenID Connect without Kibana documentation to the best of my knowledge. However, this and the openID documentation on ES revolves around a redirect uri and external web app. I'm stuck on this because the intention here is to have, among other things, back-end code authenticate against Keycloak as a requirement for hitting ES (with no web apps involved).
My docker-compose file:
version: "2.2"
services:
setup:
image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
volumes:
- certs:/usr/share/elasticsearch/config/certs
user: "0"
command: >
bash -c '
if [ x${ELASTIC_PASSWORD} == x ]; then
echo "Set the ELASTIC_PASSWORD environment variable in the .env file";
exit 1;
elif [ x${KIBANA_PASSWORD} == x ]; then
echo "Set the KIBANA_PASSWORD environment variable in the .env file";
exit 1;
fi;
if [ ! -f config/certs/ca.zip ]; then
echo "Creating CA";
bin/elasticsearch-certutil ca --silent --pem -out config/certs/ca.zip;
unzip config/certs/ca.zip -d config/certs;
fi;
if [ ! -f config/certs/certs.zip ]; then
echo "Creating certs";
echo -ne \
"instances:\n"\
" - name: es01\n"\
" dns:\n"\
" - es01\n"\
" - localhost\n"\
" ip:\n"\
" - 127.0.0.1\n"\
> config/certs/instances.yml;
bin/elasticsearch-certutil cert --silent --pem -out config/certs/certs.zip --in config/certs/instances.yml --ca-cert config/certs/ca/ca.crt --ca-key config/certs/ca/ca.key;
unzip config/certs/certs.zip -d config/certs;
fi;
echo "Setting file permissions"
chown -R root:root config/certs;
find . -type d -exec chmod 750 \{\} \;;
find . -type f -exec chmod 640 \{\} \;;
echo "Waiting for Elasticsearch availability";
until curl -s --cacert config/certs/ca/ca.crt https://es01:9200 | grep -q "missing authentication credentials"; do sleep 30; done;
echo "Setting kibana_system password";
until curl -s -X POST --cacert config/certs/ca/ca.crt -u "elastic:${ELASTIC_PASSWORD}" -H "Content-Type: application/json" https://es01:9200/_security/user/kibana_system/_password -d "{\"password\":\"${KIBANA_PASSWORD}\"}" | grep -q "^{}"; do sleep 10; done;
echo "All done!";
'
healthcheck:
test: ["CMD-SHELL", "[ -f config/certs/es01/es01.crt ]"]
interval: 1s
timeout: 5s
retries: 120
es01:
depends_on:
setup:
condition: service_healthy
image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
volumes:
- certs:/usr/share/elasticsearch/config/certs
- esdata01:/usr/share/elasticsearch/data
- type: bind
source: ./elasticsearch.yml
target: /usr/share/elasticsearch/config/elasticsearch.yml
read_only: true
- type: bind
source: ./certs.json
target: /usr/share/elasticsearch/config/certs.json
read_only: true
ports:
- ${ES_PORT}:9200
environment:
- node.name=es01
- cluster.name=${CLUSTER_NAME}
- cluster.initial_master_nodes=es01
# - cluster.initial_master_nodes=es01,es02,es03
# - discovery.seed_hosts=es02,es03
- ELASTIC_PASSWORD=${ELASTIC_PASSWORD}
- bootstrap.memory_lock=true
- xpack.security.enabled=true
- xpack.security.http.ssl.enabled=true
- xpack.security.http.ssl.key=certs/es01/es01.key
- xpack.security.http.ssl.certificate=certs/es01/es01.crt
- xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt
- xpack.security.http.ssl.verification_mode=certificate
- xpack.security.transport.ssl.enabled=true
- xpack.security.transport.ssl.key=certs/es01/es01.key
- xpack.security.transport.ssl.certificate=certs/es01/es01.crt
- xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt
- xpack.security.transport.ssl.verification_mode=certificate
- xpack.license.self_generated.type=${LICENSE}
mem_limit: ${MEM_LIMIT}
ulimits:
memlock:
soft: -1
hard: -1
healthcheck:
test:
[
"CMD-SHELL",
"curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",
]
interval: 10s
timeout: 10s
retries: 120
volumes:
certs:
driver: local
esdata01:
driver: local
.env
# Password for the 'elastic' user (at least 6 characters)
ELASTIC_PASSWORD=password
# Password for the 'kibana_system' user (at least 6 characters)
KIBANA_PASSWORD=password
# Version of Elastic products
STACK_VERSION=8.5.3
# Set the cluster name
CLUSTER_NAME=docker-cluster
# Set to 'basic' or 'trial' to automatically start the 30-day trial
# LICENSE=basic
LICENSE=trial
# Port to expose Elasticsearch HTTP API to the host
ES_PORT=9200
#ES_PORT=127.0.0.1:9200
# Port to expose Kibana to the host
KIBANA_PORT=5601
#KIBANA_PORT=80
# Increase or decrease based on the available host memory (in bytes)
MEM_LIMIT=1073741824
# Project namespace (defaults to the current folder name if not set)
#COMPOSE_PROJECT_NAME=myproject
Elasticsearch.yml
cluster.name: "docker-cluster"
network.host: 0.0.0.0
# xpack.security.authc.token.enabled: true
# xpack.security.authc.realms.oidc.oidc1:
# order: 2
# rp.client_id: "elasticsearch"
# rp.response_type: code
# rp.redirect_uri: "https://127.0.0.1:9200"
# op.issuer: "http://localhost:8080/auth/realms/Elastic"
# op.authorization_endpoint: "http://localhost:8080/auth/realms/Elastic/protocol/openid-connect/auth"
# op.token_endpoint: "http://localhost:8080/auth/realms/Elastic/protocol/openid-connect/token"
# op.jwkset_path: "certs.json"
# op.userinfo_endpoint: "http://localhost:8080/auth/realms/Elastic/protocol/openid-connect/userinfo"
# op.endsession_endpoint: "http://localhost:8080/auth/realms/Elastic/protocol/openid-connect/logout"
# rp.post_logout_redirect_uri: "https://127.0.0.1:9200/"
# claims.principal: preferred_username
My keycloak instance is running at http://localhost:8080
with a realm "Elastic" and client "elasticsearch"
So far my process is:
-
run
docker-compose up -d -
Inside the elasticsearch container:
-
Run
./bin/elasticsearch-keystore add xpack.security.authc.realms.oidc.oidc1.rp.client_secretand add the client secret under the Keycloak Realm's Client's Credentials section. -
Uncomment all the lines in elasticsearch.yml
-
-
Restart ES container
Then, following the OpenID Connect without Kibana steps, I:
-
Create the facilitator role and "facilitator" user in ES as per the Service Account user for accessing the APIs
-
Execute Step 1 of Handling the Authentication Flow section which returns a response example:
{
"redirect": "http://localhost:8080/auth/realms/Elastic/protocol/openid-connect/auth?response_type=code&redirect_uri=https%3A%2F%2F127.0.0.1%3A9200&state=Hw-RDYrwwFFUYw_9fGFYWsXyWEReCjlkdIMfKz2uBYY&nonce=jerbpKm5n2i48OGS-z5H47oTXcEL33bSOKSrfRlPQGY&client_id=elasticsearch&scope=openid",
"state": "Hw-RDYrwwFFUYw_9fGFYWsXyWEReCjlkdIMfKz2uBYY",
"nonce": "jerbpKm5n2i48OGS-z5H47oTXcEL33bSOKSrfRlPQGY",
"realm": "oidc1"
}
This is where I am unsure on how to proceed with the Authentication flow. In my elasticsearch.yml I don't know what to put as rp.redirect_uri and rp.post_logout_redirect_uri (my ES endpoint is just a filler/guess) given that there is no other web app involved. I'm sure it makes the redirect url returned invalid right now because keycloak gives a 400 error to it.