Unexpected exception while running Enterprise Search

I am trying to setup elastic enterprise search through docker compose but i keep getting ,
"Unexpected exception while running Enterprise Search" in the logs.
Here are the full logs ,

Overwriting the default Enterprise Search configuration file: /usr/share/enterprise-search/config/enterprise-search.yml (if it fails, please make sure it is writeable)
Found java executable in PATH
Java version detected: 1.8.0_252 (major version: 8)
Enterprise Search is starting...
[2021-09-03T06:33:45.061+00:00][7][2002][app-server][INFO]: Enterprise Search version=7.14.0, JRuby version=9.2.13.0, Ruby version=2.5.7, Rails version=5.1.7
[2021-09-03T06:33:45.154+00:00][7][2002][app-server][INFO]: Performing pre-flight checks for Elasticsearch running on elasticsearch:9200...
Unexpected exception while running Enterprise Search:
Error: query conflicts with opaque at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/uri/generic.rb:846:in `query='
        /usr/share/enterprise-search/lib/war/gems/gems/faraday-1.3.0/lib/faraday/connection.rb:527:in `build_exclusive_url'
        /usr/share/enterprise-search/lib/war/gems/gems/faraday-1.3.0/lib/faraday/rack_builder.rb:206:in `build_env'
        /usr/share/enterprise-search/lib/war/gems/gems/faraday-1.3.0/lib/faraday/rack_builder.rb:154:in `build_response'
        /usr/share/enterprise-search/lib/war/gems/gems/faraday-1.3.0/lib/faraday/connection.rb:492:in `run_request'
        /usr/share/enterprise-search/lib/war/gems/gems/faraday-1.3.0/lib/faraday/connection.rb:198:in `get'
        /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo/elasticsearch_checks.class:169:in `check_es_connection_with_retries!'
        /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo/elasticsearch_checks.class:34:in `check_es_connection!'
        /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo/elasticsearch_checks.class:19:in `block in run!'
        /usr/share/enterprise-search/lib/war/gems/gems/activesupport-5.1.7/lib/active_support/tagged_logging.rb:69:in `block in tagged'
        /usr/share/enterprise-search/lib/war/gems/gems/activesupport-5.1.7/lib/active_support/tagged_logging.rb:26:in `tagged'
        /usr/share/enterprise-search/lib/war/gems/gems/activesupport-5.1.7/lib/active_support/tagged_logging.rb:69:in `tagged'
        /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo/elasticsearch_checks.class:18:in `run!'
        /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo/elasticsearch_checks.class:14:in `run!'
        /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo.class:294:in `configure_elasticsearch!'
        /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo.class:273:in `configure!'
        /usr/share/enterprise-search/lib/war/config/application.class:19:in `<main>'
        org/jruby/RubyKernel.java:1016:in `load'
        /usr/share/enterprise-search/lib/war/gems/gems/activesupport-5.1.7/lib/active_support/dependencies.rb:286:in `block in load'
        /usr/share/enterprise-search/lib/war/gems/gems/activesupport-5.1.7/lib/active_support/dependencies.rb:258:in `load_dependency'
        /usr/share/enterprise-search/lib/war/gems/gems/activesupport-5.1.7/lib/active_support/dependencies.rb:286:in `load'
        /usr/share/enterprise-search/lib/war/config/application.rb:1:in `<main>'
        org/jruby/RubyKernel.java:974:in `require'
        /usr/share/enterprise-search/lib/war/gems/gems/activesupport-5.1.7/lib/active_support/dependencies.rb:292:in `block in require'
        /usr/share/enterprise-search/lib/war/gems/gems/activesupport-5.1.7/lib/active_support/dependencies.rb:258:in `load_dependency'
        /usr/share/enterprise-search/lib/war/gems/gems/activesupport-5.1.7/lib/active_support/dependencies.rb:292:in `require'
        /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo/cli/command.class:36:in `initialize'
        /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo/cli/command.class:10:in `run_and_exit'
        /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo/cli.class:143:in `run_supported_command'
        /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo/cli.class:125:in `run_command'
        /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo/cli.class:112:in `run!'
        bin/enterprise-search-internal:15:in `<main>'
Custom Enterprise Search configuration file detected, not overwriting it (any settings passed via environment will be ignored)
Found java executable in PATH
Java version detected: 1.8.0_252 (major version: 8)
Enterprise Search is starting...

Here is ent-search section from my docker-compose file ,

ent-search:
    image: docker.elastic.co/enterprise-search/enterprise-search:${ELK_VERSION}
    restart: unless-stopped
    depends_on:
      - "elasticsearch"
    environment:
      - "JAVA_OPTS=-Xmx${ELASTICSEARCH_HEAP} -Xms${ELASTICSEARCH_HEAP}"
      - "ENT_SEARCH_DEFAULT_PASSWORD=changeme"
      - "elasticsearch.username=${ELASTIC_USERNAME}"
      - "elasticsearch.password=${ELASTIC_PASSWORD}"
      - "elasticsearch.host=${ELASTICSEARCH_HOST}:${ELASTICSEARCH_PORT}"
#      - "ent.search.ssl.verificationMode= none"
#      - "app_search.listen_host=0.0.0.0"
#      - ent_search.ssl.enabled:false
      - "allow_es_settings_modification=true"
      - "secret_management.encryption_keys=[4a2cd3f81d39bf28738c10db0ca782095ffac07279561809eecc722e0c20eb09]"
      - "elasticsearch.startup_retry.interval=15"
    ports:
      - "3002:3002"
#    networks:
#      - elastic

PS: " ssl verification is ON for elasticsearch and kibana but not for ent_search (specified int he .yml file for both kibana and elasticsearch"

What does your $ELASTICSEARCH_HOST var look like? elasticsearch.host should be a full URL, including schema (http or https). Example:

export ELASTICSEARCH_HOST=http://localhost
export ELASTICSEARCH_PORT=9200

then concatenating the two would give you a full URL:

elasticsearch.host=${ELASTICSEARCH_HOST}:${ELASTICSEARCH_PORT}

comes out as:

elasticsearch.host=http://localhost:9200

Hey @Irina_Truong , I changed it to elasticsearch.host=https://localhost:9200 and got the following error in response .

Overwriting the default Enterprise Search configuration file: /usr/share/enterprise-search/config/enterprise-search.yml (if it fails, please make sure it is writeable)
Found java executable in PATH
Java version detected: 1.8.0_252 (major version: 8)
Enterprise Search is starting...
[2021-09-03T17:15:50.716+00:00][7][2002][app-server][INFO]: Enterprise Search version=7.14.0, JRuby version=9.2.13.0, Ruby version=2.5.7, Rails version=5.1.7
[2021-09-03T17:15:50.745+00:00][7][2002][app-server][INFO]: Performing pre-flight checks for Elasticsearch running on https://localhost:9200...
[2021-09-03T17:15:51.392+00:00][7][2002][app-server][WARN]: [pre-flight] Failed to connect to Elasticsearch backend. Make sure it is running and healthy.
[2021-09-03T17:17:31.402+00:00][7][2002][app-server][WARN]: [pre-flight] Failed to connect to Elasticsearch backend. Make sure it is running and healthy.

I can confirm elasticsearch is running and is kibana as well.

Ah. You get a different error! This is great. I also think my previous advice was wrong, you don't want to refer to localhost. If you are running both Elasticsearch and Enterprise Search in Docker, then you want elasticsearch.host setting to look like this - hostname elasticsearch corresponds to service name in docker-compose:

elasticsearch.host: http://elasticsearch:9200

However, if your Elasticsearch is running on localhost outside of docker, then you want the setting to look like this:

elasticsearch.host: http://host.docker.internal:9200

As far as I know, the host.docker.internal feature is available on Mac and Windows. On Linux you may need a workaround:

Hope this helps.