My Kibana is not being able to connect with my Elasticsearch because of Self-Signed Certificate

[FernandoDurier]

Good day, everyone!
Hope you are doing well.
I'm trying to connect my dockerized Kibana service with my IBM Cloud Elasticsearch.
In the past, I had no issue with another instance of Elasticsearch running inside a RHOS cluster. But now, I'm not being able to due to an SSL certificate (I guess).
In my Dockerfile I have env vars for the Elastic URL, HOST and Certificate Authorities pointing to the base64 selfsigned certificate converted to file.

Please can you help?
(The same happens to my logstash, but, I believe the solution to kibana can be used in logstash too)
Here is my Dockerfile for reference

FROM docker.elastic.co/kibana/kibana:7.6.2
ENV ELASTICSEARCH_HOSTS=$ELASTICSEARCH_HOSTS
ENV USERNAME=$USERNAME
ENV PASSWORD=$PASSWORD
ENV ELASTICSEARCH_URL=$ELASTICSEARCH_URL
COPY ./cacert.crt /share/cacert.crt
ENV ELASTICSEARCH_SSL_CA="/share/cacert.crt"
ENV XPACK_MONITORING_ELASTICSEARCH_SSL_CA="/share/cacert.crt"
ENV XPACK_SECURITY_ENABLED=true
ENV ELASTICSEARCH_SSL_CERTIFICATEAUTHORITIES="/share/cacert.crt"
ENV csp_strict=true

EXPOSE 5601 5601

[FernandoDurier]

Btw, here are the error logs in my terminal, when running the Dockerfile.

  log   [17:51:48.805] [error][admin][elasticsearch] Request error, retrying
GET https://f0a666b1-3963-4744-a7d6-f3d2b484b445.c5km1ted03t0e8geevf0.databases.appdomain.cloud:30159/_nodes?filter_path=nodes.*.version%2Cnodes.*.http.publish_address%2Cnodes.*.ip => self signed certificate in certificate chain

[FernandoDurier]

I've advanced a little now... at least changed the error and got some new extra info.

My Elasticsearch is at version 7.10.2 at IBM Cloud
I modified the Dockerfile a little (Trying to eliminate unnecessary env vars and changed the host for the one provided by IBM).

And the new error is now this:

  log   [19:13:26.521] [error][data][elasticsearch] [mapper_parsing_exception]: Failed to parse mapping [_doc]: No handler for type [flattened] declared on field [config]
  log   [19:13:26.523] [warning][savedobjects-service] Unable to connect to Elasticsearch. Error: mapper_parsing_exception
  log   [19:13:26.529] [fatal][root] { ResponseError: mapper_parsing_exception
    at IncomingMessage.response.on (/usr/share/kibana/node_modules/@elastic/elasticsearch/lib/Transport.js:272:25)
    at IncomingMessage.emit (events.js:203:15)
    at endReadableNT (_stream_readable.js:1145:12)
    at process._tickCallback (internal/process/next_tick.js:63:19)
  name: 'ResponseError',
  meta:
   { body: { error: [Object], status: 400 },
     statusCode: 400,
     headers:
      { 'content-type': 'application/json; charset=UTF-8',
        'content-length': '399' },
     meta:
      { context: null,
        request: [Object],
        name: 'elasticsearch-js',
        connection: [Object],
        attempts: 0,
        aborted: false } } }
  log   [19:13:26.543] [info][plugins-system] Stopping all plugins.

 FATAL  ResponseError: mapper_parsing_exception

Here is the modified Dockerfile:

FROM docker.elastic.co/kibana/kibana:7.10.2

COPY ./cacert.crt /share/cacert.crt

ENV XPACK_SECURITY_ENABLED=true
ENV ELASTICSEARCH_SSL_CERTIFICATEAUTHORITIES="/share/cacert.crt"

ENV ELASTICSEARCH_HOSTS=$ELASTICSEARCH_HOSTS
ENV ELASTICSEARCH_URL=$ELASTICSEARCH_URL

ENV USERNAME=$USERNAME
ENV PASSWORD=$PASSWORD

ENV csp_strict=true

EXPOSE 5601 5601

I would like to observe that when changing the version of Kibana to 7.11.2 it errors explicitly saying that it is incompatible with my cluster nodes in 7.10.2.