Enterprise Search

Elastic Enterprise Search
#1

Hi,
I setup a 3 node elasticsearch cluster on docker and enable xpack trial licence. Also setup enterprisesearch.
Enterprisesearch showing running in log side but in browser side it is not opening and showing This site can’t provide a secure connection

After running enterprisesearch getting below details:


es_enterprise       | [2021-01-28T14:49:08.823+00:00][1][2580][app-server][INFO]: Done running task: KeepFilebeatAlive
es_enterprise       | [2021-01-28T14:49:08.854+00:00][1][2582][app-server][INFO]: Done running task: RefreshFritoPieContentSources
es_enterprise       | [2021-01-28T14:49:08.855+00:00][1][2580][app-server][INFO]: Running task: RequeueStaleJobs
es_enterprise       | [2021-01-28T14:49:08.902+00:00][1][2580][cron-Work::Cron::RequeueStaleJobs][INFO]: Performing task: RequeueStaleJobs
es_enterprise       | [2021-01-28T14:49:08.934+00:00][1][2580][worker][INFO]: Updated 0 stale items from queue(s): ["connectors", "document_destroyer", "engine_destroyer", "index_adder", "indexed_doc_remover", "mailer", "refresh_document_counts", "reindexer", "schema_updater", "seed_sample_engine", "workplace_search"]
es_enterprise       | [2021-01-28T14:49:08.935+00:00][1][2580][cron-Work::Cron::RequeueStaleJobs][INFO]: Done performing task: RequeueStaleJobs
es_enterprise       | [2021-01-28T14:49:08.980+00:00][1][2580][app-server][INFO]: Done running task: RequeueStaleJobs
es_enterprise       | [2021-01-28T14:54:08.638+00:00][1][2580][app-server][INFO]: Running task: RefreshElasticsearchLicense
es_enterprise       | [2021-01-28T14:54:08.699+00:00][1][2580][cron-Work::Cron::RefreshElasticsearchLicense][INFO]: Performing task: RefreshElasticsearchLicense
es_enterprise       | [2021-01-28T14:54:08.719+00:00][1][2580][cron-Work::Cron::RefreshElasticsearchLicense][INFO]: Done performing task: RefreshElasticsearchLicense
es_enterprise       | [2021-01-28T14:54:08.745+00:00][1][2580][app-server][INFO]: Done running task: RefreshElasticsearchLicense
es_enterprise       | [2021-01-28T14:54:08.825+00:00][1][2580][app-server][INFO]: Running task: KeepFilebeatAlive
es_enterprise       | [2021-01-28T14:54:08.887+00:00][1][2580][cron-Work::Cron::KeepFilebeatAlive][INFO]: Performing task: KeepFilebeatAlive
es_enterprise       | [2021-01-28T14:54:08.888+00:00][1][2580][cron-Work::Cron::KeepFilebeatAlive][INFO]: Done performing task: KeepFilebeatAlive
es_enterprise       | [2021-01-28T14:54:08.933+00:00][1][2580][app-server][INFO]: Done running task: KeepFilebeatAlive
es_enterprise       | [2021-01-28T14:55:08.747+00:00][1][2580][app-server][INFO]: Running task: UpdateCustomSourcesConfig
es_enterprise       | [2021-01-28T14:55:08.793+00:00][1][2580][cron-Work::Cron::UpdateCustomSourcesConfig][INFO]: Performing task: UpdateCustomSourcesConfig
es_enterprise       | [2021-01-28T14:55:08.797+00:00][1][2580][cron-Work::Cron::UpdateCustomSourcesConfig][INFO]: Done performing task: UpdateCustomSourcesConfig
es_enterprise       | [2021-01-28T14:55:08.839+00:00][1][2580][app-server][INFO]: Done running task: UpdateCustomSourcesConfig
es_enterprise       | [2021-01-28T14:55:08.856+00:00][1][2580][app-server][INFO]: Running task: RefreshFritoPieContentSources
es_enterprise       | [2021-01-28T14:55:08.887+00:00][1][2580][cron-Work::Cron::RefreshFritoPieContentSources][INFO]: Performing task: RefreshFritoPieContentSources
es_enterprise       | [2021-01-28T14:55:08.891+00:00][1][2580][cron-Work::Cron::RefreshFritoPieContentSources][INFO]: Done performing task: RefreshFritoPieContentSources
es_enterprise       | [2021-01-28T14:55:08.917+00:00][1][2580][app-server][INFO]: Done running task: RefreshFritoPieContentSources
es_enterprise       | [2021-01-28T14:55:08.982+00:00][1][2580][app-server][INFO]: Running task: RequeueStaleJobs
es_enterprise       | [2021-01-28T14:55:09.027+00:00][1][2580][cron-Work::Cron::RequeueStaleJobs][INFO]: Performing task: RequeueStaleJobs
es_enterprise       | [2021-01-28T14:55:09.035+00:00][1][2580][worker][INFO]: Updated 0 stale items from queue(s): ["connectors", "document_destroyer", "engine_destroyer", "index_adder", "indexed_doc_remover", "mailer", "refresh_document_counts", "reindexer", "schema_updater", "seed_sample_engine", "workplace_search"]
es_enterprise       | [2021-01-28T14:55:09.035+00:00][1][2580][cron-Work::Cron::RequeueStaleJobs][INFO]: Done performing task: RequeueStaleJobs
es_enterprise       | [2021-01-28T14:55:09.059+00:00][1][2580][app-server][INFO]: Done running task: RequeueStaleJobs

my configuration setting is:

 enterprisesearch:
    image: docker.elastic.co/enterprise-search/enterprise-search:$ELK_VERSION
    container_name: es_enterprise
    environment:
      - cluster.name=es-docker-cluster
      - node.name=enterprisesearch
      - elasticsearch.host=https://es01:9200
      - ent_search.auth.source=standard
      - elasticsearch.username=myelasticuser
      - elasticsearch.password=$ELASTIC_PASSWORD
      - allow_es_settings_modification=true
      - ent_search.external_url=https://myservername.com:3002
      - secret_management.encryption_keys=[q2cs0f128f730y3148fa137e6cc06f3617d20e170c93a11146f448e9w97fa0cf]
      - ENT_SEARCH_DEFAULT_PASSWORD=$ELASTIC_PASSWORD
      - elasticsearch.ssl.enabled=true
      - elasticsearch.ssl.verify=false
      - "JAVA_OPTS=-Xms2g -Xmx2g"
    ports:
      - "3002:3002"
    links:
      - es01
    depends_on:
      - es01
    networks:
      - elastic
#2

Hi @vikram_singh

I think the issue is that you've configured your ent_search.external_url to use SSL (https://), but you have not set the relevant configurations to enable SSL: https://www.elastic.co/guide/en/enterprise-search/current/configure-ssl-tls.html

You can either:

  1. change to http from https like:
 - ent_search.external_url=http://myservername.com:3002
  1. enable SSL with:
ent_search.ssl.enabled: true
ent_search.ssl.keystore.path: "/path/to/keystore.jks" # modify this for the right path
ent_search.ssl.keystore.password: "changeme" # modify this for the right password
ent_search.ssl.keystore.key_password: "changeme" # modify this for the right password

Let us know if you run into other issues!

#3

Hi,

I choose 2nd option and generate keystore.jks file by command:
keytool -genkey -alias server-alias -keyalg RSA -storepass changeme -keypass changeme -keystore keystore.jks -dname 'CN=localhost, OU=Unknown, O=Unknown, L=Unknown, ST=Unknown, C=Unknown'
from elasticsearch guide book
https://www.elastic.co/guide/en/enterprise-search/current/configure-ssl-tls.html

And update my configuration

 enterprisesearch:
    image: docker.elastic.co/enterprise-search/enterprise-search:$ELK_VERSION
    container_name: es_enterprise
    environment:
      - cluster.name=es-docker-cluster
      - node.name=enterprisesearch
      - elasticsearch.host=https://es01:9200
      - ent_search.auth.source=standard
      - elasticsearch.username=elastic
      - elasticsearch.password=$ELASTIC_PASSWORD
      - allow_es_settings_modification=true
      - ent_search.external_url=https://myservername.com:3002
      - secret_management.encryption_keys=[q2cs0f128f730y3148fa137e6cc06f3617d20e170c93a11146f448e9w97fa0cf]
      - ENT_SEARCH_DEFAULT_PASSWORD=$ELASTIC_PASSWORD
      - elasticsearch.ssl.enabled=true
      - elasticsearch.ssl.verify=false
      - "JAVA_OPTS=-Xms2g -Xmx2g"
	  - ent_search.ssl.enabled=true
      - ent_search.ssl.keystore.path=keystore.jks
      - ent_search.ssl.keystore.password=changeme
      - ent_search.ssl.keystore.key_password=changeme
    ports:
      - "3002:3002"
    links:
      - es01
    depends_on:
      - es01
    networks:
      - elastic

but getting keystore.jks file error:

es01 is up-to-date
Recreating es_enterprise ... done
Attaching to es_enterprise
es_enterprise       | Found java executable in PATH
es_enterprise       | Java version detected: 1.8.0_252 (major version: 8)
es_enterprise       | Enterprise Search is starting...
es_enterprise       | *** [DEPRECATION WARNING] The setting '#/ent_search/auth/source' is deprecated and will be removed in version '8.0.0'. Please use the new auth config format ent_search.auth.<auth_name>.source.
es_enterprise       |
es_enterprise       | --------------------------------------------------------------------------------
es_enterprise       |
es_enterprise       | Invalid config file (/usr/share/enterprise-search/config/enterprise-search.yml):
es_enterprise       | The setting '#/ent_search/ssl/keystore/path' is not valid: error reading file 'keystore.jks'
es_enterprise       |
es_enterprise       | --------------------------------------------------------------------------------
es_enterprise       |
es_enterprise exited with code 1
#4

@vikram_singh can you ssh into that docker container and check that the keystore.jks file is there? I don't see that you've done anything to add a volume to your container.

If it is there, verify that the right user has read permissions for it? And if the permissions are sufficient, try using an absolute path, instead of a relative one?

#5

Hi Sean,
After running this command
keytool -genkey -alias server-alias -keyalg RSA -storepass changeme -keypass changeme -keystore keystore.jks -dname 'CN=localhost, OU=Unknown, O=Unknown, L=Unknown, ST=Unknown, C=Unknown
A file name keystore.jks is created and I give it's path in configuration file (name is elastic-docker-tls.yml).

For file permission, current user is elasticuser. Below screen keystore.jks file is present and my configuration file is elastic-docker-tls.yml. I think file permission is correct.

[elasticuser@elasticsearch-centos docker-elk-masterxpack]$ ls -l
-rw-rw-r--. 1 elasticuser elasticuser  589 Jan 19 10:11 create-certs.yml
-rw-rw-r--. 1 elasticuser elasticuser 6313 Jan 29 12:12 elastic-docker-tls.yml
drwxrwxr-x. 5 elasticuser elasticuser   69 Jan 19 09:37 elasticsearch
-rw-rw-r--. 1 elasticuser elasticuser 2611 Jan 29 12:06 keystore.jks
drwxrwxr-x. 3 elasticuser elasticuser   38 Jan 28 10:55 kibana
[elasticuser@elasticsearch-centos docker-elk-masterxpack]$
#6

Hi Sean,
For absolute path, I also tried it but getting same error.

Error:

[elasticuser@elasticsearch-centos docker-elk-masterxpack]$ docker-compose -f elastic-docker-tls.yml up enterprisesearch
es01 is up-to-date
Recreating es_enterprise ... done
Attaching to es_enterprise
es_enterprise       | Found java executable in PATH
es_enterprise       | Java version detected: 1.8.0_252 (major version: 8)
es_enterprise       | Enterprise Search is starting...
es_enterprise       | *** [DEPRECATION WARNING] The setting '#/ent_search/auth/source' is deprecated and will be removed in version '8.0.0'. Please use the new auth config format ent_search.auth.<auth_name>.source.
es_enterprise       |
es_enterprise       | --------------------------------------------------------------------------------
es_enterprise       |
es_enterprise       | Invalid config file (/usr/share/enterprise-search/config/enterprise-search.yml):
es_enterprise       | The setting '#/ent_search/ssl/keystore/path' is not valid: error reading file '/home/elasticuser/mytest/docker-elk-masterxpack/keystore.jks'
es_enterprise       |
es_enterprise       | --------------------------------------------------------------------------------
es_enterprise       |
es_enterprise exited with code 1
[elasticuser@elasticsearch-centos docker-elk-masterxpack]$

And configuration is:

environment:
      - cluster.name=es-docker-cluster
      - node.name=enterprisesearch
      - elasticsearch.host=https://es01:9200
      - ent_search.auth.source=standard
      - elasticsearch.username=elastic
      - elasticsearch.password=$ELASTIC_PASSWORD
      - allow_es_settings_modification=true
      - ent_search.external_url=https://myservername.com:3002
      - secret_management.encryption_keys=[q2cs0f128f730y3148fa137e6cc06f3617d20e170c93a11146f448e9w97fa0cf]
      - ENT_SEARCH_DEFAULT_PASSWORD=$ELASTIC_PASSWORD
      - elasticsearch.ssl.enabled=true
      - elasticsearch.ssl.verify=false
      - "JAVA_OPTS=-Xms2g -Xmx2g"
	  - ent_search.ssl.enabled=true
      - ent_search.ssl.keystore.path=/home/elasticuser/mytest/docker-elk-masterxpack/keystore.jks
      - ent_search.ssl.keystore.password=changeme
      - ent_search.ssl.keystore.key_password=changeme
    ports:
      - "3002:3002"
    links:
      - es01
    depends_on:
      - es01
    networks:
      - elastic
#7

Hi @vikram_singh ,

I think that the issue is that that file isn't on the docker container, it's in the working directory where your docker compose file is. The docker container doesn't automatically share a filesystem with the system that docker is mounted on. Some posts that might help explain this:

I'm not a docker expert, and am not as familiar with using docker compose. But when I've used docker run to add a volume, it's been like:

docker run -p 3002:3002 --name=workplace \
-e elasticsearch.host='http://host.docker.internal:9200' \
-e elasticsearch.username=elastic \
-e elasticsearch.password=changeme \
-e allow_es_settings_modification=true \
-e secret_management.encryption_keys='[4a2cd3f81d39bf28738c10db0ca782095ffac07279561809eecc722e0c20eb09]' \
-e ENT_SEARCH_DEFAULT_PASSWORD=changeme \
-e ent_search.listen_port=3002 \
-e ent_search.external_url='http://localhost:3002' \
-e ent_search.ssl.enabled=true \
-e ent_search.ssl.keystore.path=/usr/share/enterprise-search/keystore.jks \
-e ent_search.ssl.keystore.password=changeme \
-e ent_search.ssl.keystore.key_password=changeme \
-v /home/elasticuser/mytest/docker-elk-masterxpack:/usr/share/enterprise-search \
docker.elastic.co/enterprise-search/enterprise-search:7.10.2

notice in particular here the /usr/share/enterprise-search is where Enterprise Search is installed on the docker container's filesystem. So:

-e ent_search.ssl.keystore.path=/usr/share/enterprise-search/keystore.jks \

is a path to a file on the container (not on the host machine's filesystem), and

-v /home/elasticuser/mytest/docker-elk-masterxpack:/usr/share/enterprise-search \

tells docker that the volume should map the local /home/elasticuser/mytest/docker-elk-masterxpack to the container /usr/share/enterprise-search.
Note that I don't know what files you have in /home/elasticuser/mytest/docker-elk-masterxpack other than the java keystore file, so you might want to give it its own directory if you don't want to mount a ton of other files.

Give those articles a read, and let me know if you're still having issues. We're going to get this working for you, I'm confident. :slight_smile:

#8

Hi Sean,
Thanks it's working.

I change my configuration file for keystore path
- ent_search.ssl.keystore.path=/usr/share/enterprisesearch/keystore.jks

and add volumes line
volumes: - /home/elasticuser/mytest/docker-elk-masterxpack:/usr/share/enterprisesearch

Before doing above settings I copy keystore file from my docker-elk-masterxpack folder to /usr/share/enterprisesearch by
cp keystore.kjs /usr/share/enterprisesearch

Now my configuration is:

environment:
      - cluster.name=es-docker-cluster
      - node.name=enterprisesearch
      - elasticsearch.host=https://es01:9200
      - ent_search.auth.source=standard
      - elasticsearch.username=elastic
      - elasticsearch.password=$ELASTIC_PASSWORD
      - allow_es_settings_modification=true
      - ent_search.external_url=https://myservername.com:3002
      - secret_management.encryption_keys=[q2cs0f128f730y3148fa137e6cc06f3617d20e170c93a11146f448e9w97fa0cf]
      - ENT_SEARCH_DEFAULT_PASSWORD=$ELASTIC_PASSWORD
      - elasticsearch.ssl.enabled=true
      - elasticsearch.ssl.verify=false
      - "JAVA_OPTS=-Xms2g -Xmx2g"
	  - ent_search.ssl.enabled=true
      - ent_search.ssl.keystore.path=/usr/share/enterprisesearch/keystore.jks
      - ent_search.ssl.keystore.password=changeme
      - ent_search.ssl.keystore.key_password=changeme
    ports:
      - "3002:3002"
    volumes:
      - /home/elasticuser/mytest/docker-elk-masterxpack:/usr/share/enterprisesearch
    links:
      - es01
    depends_on:
      - es01
    networks:
      - elastic

Now I am configuring kibana for enterprise search.
I update my configuration
from
- ent_search.auth.source=standard to
- ent_search.auth.source=elasticsearch-native

and also update my kibana.yml for enterpriseSearch.host

Below is kibana.yml settings

#
server.name: localhost
server.host: 0.0.0.0
elasticsearch.hosts: [ "https://es01:9200" ]
#monitoring.ui.container.elasticsearch.enabled: true

# For disable sandbox error
xpack.reporting.capture.browser.chromium.disableSandbox: false

# For enterprisesearch
enterpriseSearch.host: 'https://localhost:3002'

## X-Pack security credentials
#
elasticsearch.username: elastic
elasticsearch.password: $ELASTIC_PASSWORD

But now, when I open kibana it is showing
Unable to connect
We can’t establish a connection to Enterprise Search at the host URL: https://localhost:3002

I also tried with enterpriseSearch.host: 'https://myservername.com:3002'
But no change in error.

#9

Docker is weird about localhost. Assuming that both your elasticsearch docker container and your enterprise search docker container are running on the same host, you should be able to use:

enterpriseSearch.host: 'https://host.docker.internal:3002'