"environment: - xpack.security.enabled=true" does not work with docker-compose

Elastic Stack Elasticsearch
,
1

Hi, I'm trying to specify xpack.security.enabled as an environment variable for the Elasticsearch container in the docker-compose configuration of T-Pot, a honeypot framework.
However, the following statement does not work. (This setting is reflected as an environment variable in the container. See the reproduction procedure.)

    environment:
      - discovery.type=single-node
      - xpack.security.enabled=true

Expected behavior:

$ telnet localhost 64298 (ports: - "127.0.0.1:64298:9200")
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
GET / HTTP/1.0

HTTP/1.0 401 Unauthorized
WWW-Authenticate: Basic realm="security" charset="UTF-8"
content-type: application/json; charset=UTF-8
content-length: 381

{"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}}],"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}},"status":401}Connection closed by foreign host.

Actual behavior:

$ telnet localhost 64298 (ports: - "127.0.0.1:64298:9200")
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
GET / HTTP/1.0

HTTP/1.0 200 OK
X-elastic-product: Elasticsearch
content-type: application/json; charset=UTF-8
content-length: 546

{
  "name" : "tpotcluster-node-01",
  "cluster_name" : "tpotcluster",
  "cluster_uuid" : "vFHqJ5v5RyO1_nv_0PNPiA",
  "version" : {
    "number" : "7.15.1",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "83c34f456ae29d60e94d886e455e6a3409bba9ed",
    "build_date" : "2021-10-07T21:56:19.031608185Z",
    "build_snapshot" : false,
    "lucene_version" : "8.9.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}
Connection closed by foreign host.

The reproduction procedure is as follows.

(Install T-Pot on VirtualBox with tpot.iso, and login to T-Pot virtual machine.)
PS C:\Users\tadashi.oya> ssh -l tsec -p 64295 192.168.56.102
tsec@192.168.56.102's password:
Linux cruelspike 4.19.0-18-amd64 #1 SMP Debian 4.19.208-1 (2021-09-29) x86_64
Last login: Mon Nov 29 01:35:02 2021 from 192.168.56.1
[tsec@cruelspike:~]$ sudo su -
[sudo] password for tsec:
[root@cruelspike:~]#

[root@cruelspike:~]# systemctl status tpot
...(snip)...
   Loaded: loaded (/etc/systemd/system/tpot.service; enabled; vendor preset: enabled)
...(snip)...

[root@cruelspike:~]# cat /etc/systemd/system/tpot.service
...(snip)...
# Compose T-Pot up
ExecStart=/usr/bin/docker-compose -f /opt/tpot/etc/tpot.yml up --no-color
...(snip)...

[root@cruelspike:~]# ls -l /opt/tpot/etc/tpot.yml
lrwxrwxrwx 1 root root 34 Nov 18 08:54 /opt/tpot/etc/tpot.yml -> /opt/tpot/etc/compose/standard.yml

[root@cruelspike:~]# cp /opt/tpot/etc/compose/standard.yml ~/

[root@cruelspike:~]# vi /opt/tpot/etc/compose/standard.yml
(add xpack.security.enabled and discovery.type)

[root@cruelspike:~]# diff -10 -u ~/standard.yml /opt/tpot/etc/compose/standard.yml
--- /root/standard.yml  2021-11-29 02:38:24.236000000 +0000
+++ /opt/tpot/etc/compose/standard.yml  2021-11-29 02:41:39.188000000 +0000
@@ -505,20 +505,22 @@

 #### ELK
 ## Elasticsearch service
   elasticsearch:
     container_name: elasticsearch
     restart: always
     environment:
      - bootstrap.memory_lock=true
 #     - ES_JAVA_OPTS=-Xms2048m -Xmx2048m
      - ES_TMPDIR=/tmp
+     - discovery.type=single-node
+     - xpack.security.enabled=true
     cap_add:
      - IPC_LOCK
     ulimits:
       memlock:
         soft: -1
         hard: -1
       nofile:
         soft: 65536
         hard: 65536
 #    mem_limit: 4g

[root@cruelspike:~]# systemctl stop tpot
[root@cruelspike:~]# systemctl start tpot

[root@cruelspike:~]# /opt/tpot/bin/dps.sh 15
...(snip)...
elasticsearch         Up 6 minutes (healthy)   127.0.0.1:64298->9200/tcp
...(snip)...

[root@cruelspike:~]# docker exec -it elasticsearch sh

/ $ env
HOSTNAME=2c8b01c29fee
ES_JAVA_HOME=/usr/lib/jvm/java-16-openjdk
SHLVL=1
bootstrap.memory_lock=true
HOME=/home/elasticsearch
xpack.security.enabled=true
TERM=xterm
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
ES_TMPDIR=/tmp
ES_VER=7.15.1
PWD=/
discovery.type=single-node

/ $ id
uid=2000(elasticsearch) gid=2000(elasticsearch)

/ $ ps
PID   USER     TIME  COMMAND
    1 elastics  1:13 /usr/lib/jvm/java-16-openjdk/bin/java -Xshare:auto -Des.networkaddress.cache.ttl=60 -Des.networkad
  259 elastics  0:00 sh
  324 elastics  0:00 ps

[root@cruelspike:~]# telnet localhost 64298 (ports: - "127.0.0.1:64298:9200")
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
GET / HTTP/1.0

HTTP/1.0 200 OK
X-elastic-product: Elasticsearch
content-type: application/json; charset=UTF-8
content-length: 546

{
  "name" : "tpotcluster-node-01",
  "cluster_name" : "tpotcluster",
  "cluster_uuid" : "vFHqJ5v5RyO1_nv_0PNPiA",
  "version" : {
    "number" : "7.15.1",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "83c34f456ae29d60e94d886e455e6a3409bba9ed",
    "build_date" : "2021-10-07T21:56:19.031608185Z",
    "build_snapshot" : false,
    "lucene_version" : "8.9.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}
Connection closed by foreign host.

Thanks.

2

tpot's standard.yml doesn't use the official Elasticsearch image.

I believe the image it relies on is this one: tpotce/Dockerfile at master · telekom-security/tpotce · GitHub which installs Elasticsearch from a tarball, and doesn't have the same support for configuration via docker environment variables.

You'll either need to take this up with them, or switch to using the official Elasticsearch Docker image.

3

Thank you for pointing that out.
I hadn't thought about the fact that Elasticsearch from a tarball doesn't support environment variables.
I have confirmed the following at hand.

$ wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.15.2-linux-x86_64.tar.gz
$ tar xvzf elasticsearch-7.15.2-linux-x86_64.tar.gz
$ cd elasticsearch-7.15.2
$ env xpack.security.enabled=true ./bin/elasticsearch

$ telnet localhost 9200
Trying ::1...
Connected to localhost.
Escape character is '^]'.
GET / HTTP/1.0

HTTP/1.0 200 OK
X-elastic-product: Elasticsearch
Warning: 299 Elasticsearch-7.15.2-93d5a7f6192e8a1a12e154a2b81bf6fa7309da0c "Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.15/security-minimal-setup.html to enable security."
content-type: application/json; charset=UTF-8
content-length: 532

{
  "name" : "elk",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "dDhohkk4RK6et4Syj2zuEw",
  "version" : {
    "number" : "7.15.2",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "93d5a7f6192e8a1a12e154a2b81bf6fa7309da0c",
    "build_date" : "2021-11-04T14:04:42.515624022Z",
    "build_snapshot" : false,
    "lucene_version" : "8.9.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}
Connection closed by foreign host.

In the T-Pot configuration, it worked by mounting Elasticsearch.yml with docker-compose as shown below.

# vi /path/to/elasticsearch.yml
cluster.name: tpotcluster
node.name: "tpotcluster-node-01"
xpack.ml.enabled: false
xpack.security.enabled: true <<<HERE
#xpack.ilm.enabled: false
path:
    logs: /data/elk/log
    data: /data/elk/data
http.host: 0.0.0.0
http.cors.enabled: true
http.cors.allow-origin: "*"
indices.query.bool.max_clause_count: 2000
cluster.initial_master_nodes:
- "tpotcluster-node-01"
discovery.zen.ping.unicast.hosts:
- localhost

# vi /opt/tpot/etc/tpot.yml
[...]
#### ELK
## Elasticsearch service
  elasticsearch:
    container_name: elasticsearch
    restart: always
    environment:
     - bootstrap.memory_lock=true
#     - ES_JAVA_OPTS=-Xms2048m -Xmx2048m
     - ES_TMPDIR=/tmp
    cap_add:
     - IPC_LOCK
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536
        hard: 65536
#    mem_limit: 4g
    ports:
     - "127.0.0.1:64298:9200"
    image: "dtagdevsec/elasticsearch:2006"
    volumes:
     - /data:/data
     - /path/to/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml <<<HERE
[...]

# systemctl start tpot

And I got this as I expected:

$ telnet localhost 64298
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
GET / HTTP/1.0

HTTP/1.0 401 Unauthorized
WWW-Authenticate: Basic realm="security" charset="UTF-8"
content-type: application/json; charset=UTF-8
content-length: 381

{"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}}],"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}},"status":401}Connection closed by foreign host.

Thanks.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.