APM Server with docker - Only POST requests are supported

[massilva (Marcos Antônio De Souza Silva)]

If you are asking about a problem you are experiencing, please use the following template, as it will help us help you. If you have a different problem, please delete all of this text

Kibana version:
7.0.0

Elasticsearch version:
7.0.0

APM Server version:
7.0.0

APM Agent language and version:
Python (Django)
4.2.2

Browser version:
"user-agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.108 Safari/537.36"

Original install method (e.g. download page, yum, deb, from source, etc.) and version: Docker

Is there anything special in your setup?
Using docker-compose, kibana, elasticsearch, apm and django

Description of the problem including expected versus actual behavior. Please include screenshots (if relevant):

The application (Django) cannot submit data to APM Server.

Steps to reproduce

1. Run APM-SERVER with docker-compose

apm-server.yml

# https://github.com/elastic/apm-server/blob/master/apm-server.yml
apm-server:
  host: "0.0.0.0:8200"
  secret: "d910fe18cd0fde99ee282685cc7046e14a8491df474b85c8"


setup.kibana:
  host: "kibana:5601"
  username: "elastic"
  password: "changeme"


output.elasticsearch:
  hosts: ["elasticsearch:9200"]
  username: "elastic"
  password: "changeme"
  indices:
    - index: "apm-%{[beat.version]}-sourcemap"
      when.contains:
        processor.event: "sourcemap"

    - index: "apm-%{[beat.version]}-error-%{+yyyy.MM.dd}"
      when.contains:
        processor.event: "error"

    - index: "apm-%{[beat.version]}-transaction-%{+yyyy.MM.dd}"
      when.contains:
        processor.event: "transaction"

    - index: "apm-%{[beat.version]}-span-%{+yyyy.MM.dd}"
      when.contains:
        processor.event: "span"

apm-server.rum.enabled: true
apm-server.rum.event_rate.limit: 300
apm-server.rum.event_rate.lru_size: 1000
apm-server.rum.allow_origins: ['*']
apm-server.rum.library_pattern: "node_modules|bower_components|~"
apm-server.rum.exclude_from_grouping: "^/webpack"
apm-server.rum.source_mapping.cache.expiration: 5m
apm-server.rum.source_mapping.index_pattern: "apm-*-sourcemap*"

kibana.yml

server:
  name: "kibana"
  host: "0.0.0.0"

elasticsearch:
    #url: "http://elasticsearch:9200"
  hosts: "http://elasticsearch:9200"
  username: "elastic"
  password: "changeme"

Dockerfile

FROM docker.elastic.co/apm/apm-server:7.0.0
RUN mkdir -p /usr/share/apm-server
COPY ./apm-server.yml /usr/share/apm-server/apm-server.yml
USER root
RUN chown apm-server /usr/share/apm-server/apm-server.yml
USER apm-server

docker-compose.yml

version: "2.2"
services:
  # https://www.elastic.co/guide/en/apm/server/current/running-on-docker.html
  apm-server:
    #image: docker.elastic.co/apm/apm-server:6.2.3
    build:
        context: .
    container_name: apm-server
    ports:
      - 8200:8200
    networks:
      - apm-net
    volumes:
      - "./apm-server.yml:/usr/share/apm-server/apm-server.yml"
    depends_on:
      elasticsearch:
        condition: service_healthy
    command: --strict.perms=false -e  # Avoid the error: "apm-server.yml" can only be writable by the owner.

  # https://www.elastic.co/guide/en/kibana/current/_configuring_kibana_on_docker.html
  kibana:
    image: docker.elastic.co/kibana/kibana:7.0.0
    container_name: apm-kibana
    ports:
      - 5601:5601
    networks:
      - apm-net
    volumes:
      - "./kibana.yml:/usr/share/kibana/config/kibana.yml"
    depends_on:
      elasticsearch:
        condition: service_healthy

  # https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.0.0
    container_name: apm-es
    environment:
      - "ELASTIC_PASSWORD=changeme"
      - "http.host=0.0.0.0"
      - "transport.host=127.0.0.1"
    ports:
      - 9200:9200
    networks:
      - apm-net
    volumes:
      - "esdata:/usr/share/elasticsearch/data"
    healthcheck:
      test: ['CMD', 'curl', '-f', '-u', 'elastic:changeme', 'http://localhost:9200']

volumes:
  esdata:
    driver: local

networks:
  apm-net:

Run

docker-compose -f docker-compose.yml up -d --build

2. Configure the django application using the APM Python Agent Reference [3.x] - Django Support documentation.

Errors in browser console (if relevant):

Failed to submit message: 'Connection to APM Server timed out (url: http://localhost:8200/intake/v2/events, timeout: 5 seconds)'

dropping flushed data due to transport failure back-off

APM Server log:

2019-04-26T15:14:00.366Z	INFO	[request]	beater/intake_handler.go:96	error handling request	{"request_id": "666905ba-8058-4cb9-8a27-b1a59a65213d", "method": "GET", "URL": "/intake/v2/events", "content_length": 0, "remote_address": "172.27.0.1", "user-agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.108 Safari/537.36", "error": "only POST requests are supported"}

[axw (Andrew Wilkins)]

Hi @massilva, welcome to the forum!

The user-agent in the APM Server log record suggests that you visited "http://localhost:8200/intake/v2/events" in Safari, which will use the GET method and therefore log that message. This is not particularly relevant to the agent's inability to connect, although it does show that it's accepting connections.

Where is your Django application running? Is it inside Docker? Assuming it is, you will need to update your Django settings with the APM Server URL, per https://www.elastic.co/guide/en/apm/agent/python/current/configuration.html#django-configuration. e.g.

ELASTIC_APM = {
    'SERVICE_NAME': 'my-app',
    'SECRET_TOKEN': 'changeme',
    'SERVER_URL': 'http://apm-server:8200',
}

P.S. You pasted the APM Server secret in the forum; you may want to regenerate that now.

P.P.S. You linked to the 3.x docs for the Python agent, but say above that you're using version 4.2.2. It looks like version 4.2.2 is correct, since older versions won't use the URL logged.