Doubts to enable xpack in elasticsearch 8.x

Elastic Stack Elasticsearch
1

I'm trying to install Elasticsearch 8 with the xpack module enabled, but I'm facing the errors below when I start the service, could you help me?

Mar 22 10:35:11 elk systemd-entrypoint[748]: at com.fasterxml.jackson.dataformat.yaml.YAMLParser.nextToken(YAMLParser.java:399)
Mar 22 10:35:11 elk systemd-entrypoint[748]: at org.elasticsearch.xcontent.json.JsonXContentParser.nextToken(JsonXContentParser.java:45)
Mar 22 10:35:11 elk systemd-entrypoint[748]: at org.elasticsearch.common.settings.Settings.fromXContent(Settings.java:707)
Mar 22 10:35:11 elk systemd-entrypoint[748]: at org.elasticsearch.common.settings.Settings.fromXContent(Settings.java:676)
Mar 22 10:35:11 elk systemd-entrypoint[748]: at org.elasticsearch.common.settings.Settings$Builder.loadFromStream(Settings.java:1184)

my Elasticsearch.yml is that one

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
#cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
#node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/lib/elasticsearch
#
# Path to log files:
#
path.logs: /var/log/elasticsearch
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# By default Elasticsearch is only accessible on localhost. Set a different
# address here to expose this node on the network:
#
#network.host: 192.168.0.1
network.host: ["_eth0_", "_local_"]
#
# By default Elasticsearch listens for HTTP traffic on the first free port it
# finds starting at 9200. Set a specific HTTP port here:
#
#http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.seed_hosts: ["host1", "host2"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
#cluster.initial_master_nodes: ["node-1", "node-2"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Allow wildcard deletion of indices:
#
#action.destructive_requires_name: false

#----------------------- BEGIN SECURITY AUTO CONFIGURATION -----------------------
#
# The following settings, TLS certificates, and keys have been automatically      
# generated to configure Elasticsearch security features on 22-03-2022 13:02:59
#
# --------------------------------------------------------------------------------

# Enable security features
xpack.security.enabled: true

xpack.security.enrollment.enabled: true

# Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents
xpack.security.http.ssl:
  enabled: true
  keystore.path: certs/http.p12

# Enable encryption and mutual authentication between cluster nodes
xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/transport.p12
  truststore.path: certs/transport.p12
# Create a new cluster with the current node only
# Additional nodes can still join the cluster later
cluster.initial_master_nodes: ["elk"]

# Allow HTTP API connections from localhost and local networks
# Connections are encrypted and require user authentication
http.host: [_local_, _site_]

# Allow other nodes to join the cluster from localhost and local networks
# Connections are encrypted and mutually authenticated
#transport.host: [_local_, _site_]

#----------------------- END SECURITY AUTO CONFIGURATION -------------------------


# Add the rest of these settings at the bottom of the file
discovery.type: single-node

# Enable security
xpack.security.enabled: true

# Enable auditing if you want, uncomment
xpack.security.audit.enabled: true

# SSL HTTP Settings
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.keystore.path: http.p12

# SSL Transport Settings
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.client_authentication: required
xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: elastic-certificates.p12
2

If you want to configure security features yourself ( which looks like you do ! ), you need to adjust or remove the configuration that was automatically created for you on first run

Please remove everything between

#----------------------- BEGIN SECURITY AUTO CONFIGURATION -----------------------

and

#----------------------- END SECURITY AUTO CONFIGURATION -------------------------

and start the node again

3

@ikakavas thats works, now im with another problem.
im using the original certs that already been generated by default to Elasticsearch.
after i setup the SSL conf in my kibana.yml, the service works ok, but when i will access the web page e see the message Kibana server is not ready yet., do you know why im seeing this? my log dont say nothing.
here are my kibana.yml

# =================== System: Kibana Server (Optional) ===================
# Enables SSL and paths to the PEM-format SSL certificate and SSL key files, respectively.
# These settings enable SSL for outgoing requests from the Kibana server to the browser.
server.ssl.enabled: true
server.ssl.certificate: /etc/kibana/certs/webmais.crt
server.ssl.key: /etc/kibana/certs/webmais.key

# =================== System: Elasticsearch ===================
# The URLs of the Elasticsearch instances to use for all your queries.
elasticsearch.hosts: ["https://10.0.0.117:9200"]

# If your Elasticsearch is protected with basic authentication, these settings provide
# the username and password that the Kibana server uses to perform maintenance on the Kibana
# index at startup. Your Kibana users still need to authenticate with Elasticsearch, which
# is proxied through the Kibana server.
#elasticsearch.username: "kibana_system"
#elasticsearch.password: "pass"

# Kibana can also authenticate to Elasticsearch via "service account tokens".
# Service account tokens are Bearer style tokens that replace the traditional username/password based configuration.
# Use this token instead of a username/password.
elasticsearch.serviceAccountToken: "eyJ2ZXIiOiI4LjEuMSIsImFkciI6WyIxMC4wLjAuMTE3OjkyMDAiXSwiZmdyIjoiODg2OGIzYWQ0MThiZWZlZThhMGJjYzVmNDY1ODE0YjgzODVhZjA0MzQ4ZWUxYjIzMDEzYjJiYjA4MGYwZGZmNCIsImtleSI6Ikw5QTN2WDhCN3BfTVpjTEExei13OlNOSG1KY0YyUk9DdjVyaXNuSkRyR2cifQ=="

# Time in milliseconds to wait for Elasticsearch to respond to pings. Defaults to the value of
# the elasticsearch.requestTimeout setting.
#elasticsearch.pingTimeout: 1500

# Time in milliseconds to wait for responses from the back end or Elasticsearch. This value
# must be a positive integer.
#elasticsearch.requestTimeout: 30000

# Specifies whether Kibana should use compression for communications with elasticsearch
# Defaults to `false`.
#elasticsearch.compression: false

# List of Kibana client-side headers to send to Elasticsearch. To send *no* client-side
# headers, set this value to [] (an empty list).
#elasticsearch.requestHeadersWhitelist: [ authorization ]

# Header names and values that are sent to Elasticsearch. Any custom headers cannot be overwritten
# by client-side headers, regardless of the elasticsearch.requestHeadersWhitelist configuration.
#elasticsearch.customHeaders: {}

# Time in milliseconds for Elasticsearch to wait for responses from shards. Set to 0 to disable.
#elasticsearch.shardTimeout: 30000

# =================== System: Elasticsearch (Optional) ===================
# These files are used to verify the identity of Kibana to Elasticsearch and are required when
# xpack.security.http.ssl.client_authentication in Elasticsearch is set to required.
elasticsearch.ssl.certificate: /etc/kibana/certs/http_ca.crt
#elasticsearch.ssl.key: /etc/elasticsearch/certs/client.key

# Enables you to specify a path to the PEM file for the certificate
# authority for your Elasticsearch instance.
#elasticsearch.ssl.certificateAuthorities: [ "/path/to/your/CA.pem" ]

# To disregard the validity of SSL certificates, change this setting's value to 'none'.
elasticsearch.ssl.verificationMode: none
4

You have this configured wrong:

Should be

elasticsearch.ssl.certificateAuthorities: [ "/etc/kibana/certs/http_ca.crt" ]

Since you are using the autoconfiguration material either way, why don’t you use the actual autoconfiguration itself? All this would be taken care for you automatically without needing to touch any of the config files

5

@ikakavas i changed this but the error persist.
but, how i use auto config in Kibana? when i try to run Kibana without change nothing in kibana.yml, i see this error on the log "kibana is not configured"

6

I reinstalled the stack today without changing the Elasticsearch and kibana .yml files and now kibana displays the following message in the log

Mar 25 11:02:48 elkcomssl kibana[8633]: i Kibana has not been configured.
Mar 25 11:02:48 elkcomssl kibana[8633]: Go to http://localhost:5601/?code=505878 to get started.

but when i try to access this from my computer (not from VM) i cant access, do you know how i can access this? i already try to change to http://vmIP:5601/?code=505878

7

It’s not an error, it’s an informational message. It gives you a link and instructions to click it so that you can configure kibana in your browser.

See our docs here : Start the Elastic Stack with security enabled | Elasticsearch Guide [8.1] | Elastic

8

@ikakavas when i changed the server.host option in kibana.yml to my vm ip, i can access the webpage and configure with success the stack.

but, when i need to renew the certs, what i can do? because when i generate new certs using the recommended procedure, I had some issues that you may have seen earlier here in the thread

to generate new certs i use the Elasticsearch-certutil

9

I take it this is a new installation and you have no data to lose so I’d just start over and use the instructions from here Start the Elastic Stack with security enabled | Elasticsearch Guide [8.1] | Elastic

This is the simplest way to get you up and running, it’s designed to work out of the box

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.