POST https://apm.<acme>.com/intake/v2/rum/events net::ERR_BLOCKED_BY_CLIENT

[Kay_Khan]

I have an elastic stack on kubernetes (k8s) using ECK.

Kibana version:
7.13.2
Elasticsearch version:
7.13.2
APM Server version:
7.13.2
APM Agent language and version:
@elastic/apm-rum - npm - 5.9.1
Browser version:
Chrome latest

Description of the problem including expected versus actual behavior. Please include screenshots (if relevant):

Frontend apm-run agent fails to send messages to apm server. if i disable cors on the browser it works - google-chrome --disable-web-security --user-data-dir=temp then navigate to my frontend http://localhost:4201/

[Elastic APM] Failed sending events! Error: https://apm.<redacted>.com/intake/v2/rum/events HTTP status: 0
    at ApmServer._constructError (apm-server.js:120)
    at eval (apm-server.js:48)

POST https://apm.<acme>.com/intake/v2/rum/events net::ERR_BLOCKED_BY_CLIENT

Steps to reproduce:

apm.yml

apiVersion: apm.k8s.elastic.co/v1
kind: ApmServer
metadata:
  name: apm-server-prod
  namespace: elastic-system
spec:
  version: 7.13.2
  count: 1
  elasticsearchRef:
    name: "elasticsearch-prod"
  kibanaRef:
    name: "kibana-prod"
  http:
    service:
      spec:
        type: NodePort
  config:
    apm-server:
      rum.enabled: true
      ilm.enabled: true

elastic.ingress.yml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: elastic-ingress
  namespace: elastic-system
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/group.name: "<redacted>"
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP":80,"HTTPS": 443}]'
    alb.ingress.kubernetes.io/backend-protocol: 'HTTPS'
    alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-east-2:700849607999:certificate/0250a551-8971-468d-a483-cad28f890463
    alb.ingress.kubernetes.io/tags: Environment=prod,Team=dev
    alb.ingress.kubernetes.io/healthcheck-path: /health
    alb.ingress.kubernetes.io/healthcheck-interval-seconds: '300'
    alb.ingress.kubernetes.io/load-balancer-attributes: access_logs.s3.enabled=true,access_logs.s3.bucket=<redacted>-aws-ingress-logs,access_logs.s3.prefix=dev-ingress
spec:
  rules:
    - host: elasticsearch.<redacted>.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: elasticsearch-prod-es-http
                port:
                  number: 9200
    - host: kibana.<redacted>.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: kibana-prod-kb-http
                port:
                  number: 5601
    - host: apm.<redacted>.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: apm-server-prod-apm-http
                port:
                  number: 8200

frontend.js

import { init as initApm } from "@elastic/apm-rum";
import config from "<redacted>-web/config/environment";

export const apm = initApm({
  serviceName: "frontend",
  serverUrl: "https://apm.<redacted>.com",
  environment: config.environment,
  logLevel: "debug",
});

Errors in browser console (if relevant):

Provide logs and/or server output (if relevant):

apm-server pod does not apear to display any errors in this case, i assume the client never reaches the server .

[vigneshshanmugam]

Hi @Kay_Khan ,

It seems like some of the response headers are missing for the Preflight request.

Please check the docs here - Configure CORS | APM Real User Monitoring JavaScript Agent Reference [5.x] | Elastic

Let us know iff that helps.

Thanks,
Vignesh

[Kay_Khan]

Hi @vigneshshanmugam i saw that page but i read that it defaults to allow all origins

A list of permitted origins can be supplied to the apm-server.rum.allow_origins configuration option. By default, APM Server allows all origins.

So i did not think i needed to do anything extra? I am little confused

My frontend which has apm-run agent is on localhost:4201
My apm server is on https://apm.acme.com

[vigneshshanmugam]

Hi @Kay_Khan ,

I was thinking if your configured APM server sends the proper CORS response headers for the /events request sent from your browser.

Access-Control-Allow-Headers: Content-Type
Access-Control-Allow-Methods: POST, OPTIONS
Access-Control-Allow-Origin: [request-origin]

Do you see these for the OPTIONS request?

Thanks,
Vignesh

[Kay_Khan]

@vigneshshanmugam Can you clarify where im suppose to be looking for that? Because i cannot see it in the network tab of my browser

The frontend when looking at the network tab in chrome does not show me a request for OPTIONS /events. There is only a single request to POST /events which i have screenshotted in my OP.

[Kay_Khan]

Hello?

[vigneshshanmugam]

Apologies for the delay,

Can you disable ad blockers and extensions and try again on a fresh browser tab without any extensions to find if its an issue with CORS or with extensions?

Thanks,
Vignesh