Kibana version: 7.1.1
Elasticsearch version: 7.1.1
APM Server version: 7.1.1
APM Agent language and version:
Browser version:
Original install method (e.g. download page, yum, deb, from source, etc.) and version: yum upgrade
Fresh install or upgraded from other version? 6.8.0
Is there anything special in your setup? For example, are you using the Logstash or Kafka outputs? Are you using a load balancer in front of the APM Servers? Have you changed index pattern, generated custom templates, changed agent configuration etc.
Description of the problem including expected versus actual behavior. Please include screenshots (if relevant):
Steps to reproduce:
1.
2.
3.
Errors in browser console (if relevant):
Provide logs and/or server output (if relevant):
Description: after upgrade to 7.1.1 from 6.8.0 I don't find any services in APM.
If we look at the logfile,
{"type":"log","@timestamp":"2019-06-03T10:07:58Z","tags":["debug","legacy-proxy"],"pid":8348,"message":"Event is being forwarded: connection"}
{"type":"log","@timestamp":"2019-06-03T10:07:58Z","tags":["debug","legacy-service"],"pid":8348,"message":"Request will be handled by proxy GET:/api/apm/services?start=2019-06-02T10%3A07%3A58.495Z&end=2019-06-03T10%3A07%3A58.495Z."}
{"type":"log","@timestamp":"2019-06-03T10:07:58Z","tags":["debug","security","basic"],"pid":8348,"message":"Trying to authenticate user request to /api/apm/services?start=2019-06-02T10%3A07%3A58.495Z&end=2019-06-03T10%3A07%3A58.495Z."}
{"type":"log","@timestamp":"2019-06-03T10:07:58Z","tags":["debug","security","basic"],"pid":8348,"message":"Trying to authenticate via login attempt."}
{"type":"log","@timestamp":"2019-06-03T10:07:58Z","tags":["debug","security","basic"],"pid":8348,"message":"Username and password not found in payload."}
{"type":"log","@timestamp":"2019-06-03T10:07:58Z","tags":["debug","security","basic"],"pid":8348,"message":"Trying to authenticate via header."}
{"type":"log","@timestamp":"2019-06-03T10:07:58Z","tags":["debug","security","basic"],"pid":8348,"message":"Authorization header is not presented."}
{"type":"log","@timestamp":"2019-06-03T10:07:58Z","tags":["debug","security","basic"],"pid":8348,"message":"Trying to authenticate via state."}
{"type":"log","@timestamp":"2019-06-03T10:07:58Z","tags":["debug","security","basic"],"pid":8348,"message":"Request has been authenticated via state."}
{"type":"log","@timestamp":"2019-06-03T10:07:58Z","tags":["license","debug","xpack"],"pid":8348,"message":"Calling [data] Elasticsearch _xpack API. Polling frequency: 30001"}
{"type":"response","@timestamp":"2019-06-03T10:07:58Z","tags":,"pid":8348,"method":"get","statusCode":400,"req":{"url":"/api/apm/services?start=&end=","method":"get","headers":{"host":"kibana.intranet.cnb:6501","user-agent":"Mozilla/5.0 (Windows NT 6.1; WOW64; rv:60.0) Gecko/20100101 Firefox/60.0","accept":"/","accept-language":"de,en-US;q=0.7,en;q=0.3","accept-encoding":"gzip, deflate, br","referer":"https://kibana.intranet.cnb:6501/app/apm","content-type":"application/json","kbn-version":"7.1.1","origin":"https://kibana.intranet.cnb:6501","dnt":"1","connection":"keep-alive"},"remoteAddress":"x.x.x.x","userAgent":"x.x.x.x","referer":"https://kibana.intranet.cnb:6501/app/apm"},"res":{"statusCode":400,"responseTime":35,"contentLength":9},"message":"GET /api/apm/services?start=&end= 400 35ms - 9.0B"}
we notice that the timestamp is not forwarded in 7.1.1.
In Version 7.1.0 of another ELK-Stack it is forwarding as shown here (http 401 is returned, because APM has no data in this installation yet):
{"type":"log","@timestamp":"2019-06-03T10:00:42Z","tags":["debug","legacy-service"],"pid":55089,"message":"Request will be handled by proxy GET:/api/apm/services?start=2019-06-02T10%3A00%3A42.375Z&end=2019-06-03T10%3A00%3A42.379Z."}
{"type":"log","@timestamp":"2019-06-03T10:00:42Z","tags":["debug","security","basic"],"pid":55089,"message":"Trying to authenticate user request to /api/apm/services?start=2019-06-02T10%3A00%3A42.375Z&end=2019-06-03T10%3A00%3A42.379Z."}
{"type":"response","@timestamp":"2019-06-03T10:00:42Z","tags":,"pid":55089,"method":"get","statusCode":401,"req":{"url":"/api/apm/services?start=2019-06-02T10%3A00%3A42.375Z&end=2019-06-03T10%3A00%3A42.379Z","method":"get","headers":{"host":"x.x.x.x:6601","user-agent":"Mozilla/5.0 (Windows NT 6.1; WOW64; rv:60.0) Gecko/20100101 Firefox/60.0","accept":"/","accept-language":"de,en-US;q=0.7,en;q=0.3","accept-encoding":"gzip, deflate","referer":"http://x.x.x.x:6601/app/apm","content-type":"application/json","kbn-version":"7.1.0","origin":"http://x.x.x.x:6601","dnt":"1","connection":"keep-alive"},"remoteAddress":"x.x.x.x","userAgent":"x.x.x.x.","referer":"http://x.x.x.x:6601/app/apm"},"res":{"statusCode":401,"responseTime":7,"contentLength":9},"message":"**GET /api/apm/services?start=2019-06-02T10%3A00%3A42.375Z&end=2019-06-03T10%3A00%3A42.379Z** 401 7ms - 9.0B"}
