EOF while reading

Elastic Observability APM
7

Oh, I deeped down to the problem and created and configured roles, users and index templates for apm-server:

  • role apm-server with all (for test) privilleges for metrics-apm*, logs-apm* and traces-apm* and with monitor, cluster:admin/xpack/monitoring/bulk cluster privilleges
  • user apm-server with role apm-server
  • index templates logs-apm, metrics-apm and traces-apm configured as data streams with patterns logs-apm*, metrics-apm* and traces-apm*

It is absolutely true that my test requests are processing cause sample data are indexing. I see indices in the UI and in responses using Search API.

After several apm-server restarts I get a few errors.

IDK why I see this error. I have index template with neccessary index pattern:

Nov 29 10:22:26 fleet-server apm-server[11648]: {"log.level":"error","@timestamp":"2022-11-29T10:22:26.852Z","log.logger":"beater","log.origin":{
"file.name":"beater/waitready.go","file.line":62},"message":"precondition 'apm integration installed' failed: error querying Elasticsearch for in
tegration index templates: unexpected HTTP status: 404 Not Found ({\"error\":{\"root_cause\":[{\"type\":\"resource_not_found_exception\",\"reason
\":\"index template matching [logs-apm.error] not found\"}],\"type\":\"resource_not_found_exception\",\"reason\":\"index template matching [logs-
apm.error] not found\"},\"status\":404}): to remediate, please install the apm integration: https://ela.st/apm-integration-quickstart","service.n
ame":"apm-server","ecs.version":"1.6.0"}

And the second error:

Nov 29 10:22:26 fleet-server apm-server[11648]: {"log.level":"error","@timestamp":"2022-11-29T10:22:26.858Z","log.logger":"kibana","log.origin":{
"file.name":"kibana/connecting_client.go","file.line":79},"message":"failed to obtain connection to Kibana: fail to get the Kibana version: fail
to unmarshal the response from GET https://{KIBANA}:443/api/status. Response: <!DOCTYPE html> <html class=\"login-pf\">  <head>
     <meta charset=\"utf-8\">     <meta http-equiv=\"Content-Type\" content=\"text/html; charset=UTF-8\" />     <meta name=\"robots\" content=\"n
oindex, nofollow\">     <link rel=\"icon\" type=\"image/x-icon\" href=\".... (truncated). Kibana status api returns: invalid character '<' lookin
g for beginning of value","service.name":"apm-server","ecs.version":"1.6.0"}

This happened after I configured apm-server.kibana section in apm-server.yml. I haven't this section before so it possible that it is cause with apm-server integration installing issue. But it is not work. I think cause is enabled SSO on Kibana side. Are there ways to configure apm-server.kibana to work with SSO?