Elastic APM Server and openshift

Kibana version:
n/a

Elasticsearch version:
openshift operator 4.6.0-202112140939

APM Server version:
7.16.2

APM Agent language and version:
n/a

Browser version:
n/a

Original install method (e.g. download page, yum, deb, from source, etc.) and version:
ECK operator's APM or standalone APM server

Fresh install or upgraded from other version?
fresh install

Is there anything special in your setup?
Talking to openshift's Elasticsearch in openshift-logging

Description of the problem including expected versus actual behavior. Please include screenshots (if relevant):
I'm trying to get elastic's APM CRD or apm server to talk to openshift's Elasticsearch operator 4.6.0-202112140939. In most cases, I get a compatibility error. If I specify certain older version of APM, I get a compatibility error, which recommends using the OSS version. But the OSS version seems to still give an error. Am I missing something?

With APM server 7.16.2, I get:

{"log.level":"error","@timestamp":"2022-01-04T22:51:17.166Z","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/output.go","file.line":154},"message":"Failed to connect to backoff(elasticsearch(https://elasticsearch.openshift-logging.svc:9200)): Connection marked as failed because the onConnect callback failed: could not connect to a compatible version of Elasticsearch: 400 Bad Request: {\"error\":{\"root_cause\":[{\"type\":\"invalid_index_name_exception\",\"reason\":\"Invalid index name [_license], must not start with '_'.\",\"index_uuid\":\"_na_\",\"index\":\"_license\"}],\"type\":\"invalid_index_name_exception\",\"reason\":\"Invalid index name [_license], must not start with '_'.\",\"index_uuid\":\"_na_\",\"index\":\"_license\"},\"status\":400}","service.name":"apm-server","ecs.version":"1.6.0"}

With the APM CRD, specifying older versions, I get similar errors unless I go back to 7.12.1 or certain earlier versions. With those, I get:

{"log.level":"error","@timestamp":"2021-12-06T23:14:37.387Z","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/output.go","file.line":154},"message":"Failed to connect to backoff(elasticsearch(https://elasticsearch.openshift-logging.svc:9200)): Connection marked as failed because the onConnect callback failed: Apm-Server requires the default distribution of Elasticsearch. Please update to the default distribution of Elasticsearch for full access to all free features, or switch to the OSS distribution of Apm-Server.","ecs.version":"1.6.0"}

Is there any way to make this work? Either using the OSS or the COTS versions of the APM server? Or maybe we just can't get there from here.

Steps to reproduce:

  1. See above. Issue happens at startup.

Errors in browser console (if relevant):
n/a

Provide logs and/or server output (if relevant):
See above

Welcome to our community!

Reading the definitions that openshift operator 4.6.0-202112140939 uses, it seems that it relies on on Elasticsearch 6 in an image that is maintained by Openshift themselves.

The log message is not the most comprehensible but it is logged since Elasticsearch 6.8.x is indeed incompatible with APM Server 7.16.2. I would advise using a compatible Elasticsearch version instead.

It seems that you're already using ECK for the APM Server's CRDs, so I'd advise using a dedicated Elasticsearch deployment for APM data, since the OpenShift logging Elasticsearch cluster relies on 6.x.

Thank you, Marc.

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.