Failed to submit message: 'HTTP 401: invalid token' to APM Server

Elastic Observability APM
1

Hi
I have ELk stack and APM server (6.5.0) are running in Docker. I can view the Kibana dashboard for any events. Created a sample flask application to send APM values of it to APM server. When i call the Flask application through the following command it produces 'HTTP 401: invalid token' error.

The flask application is configured with token from APM server:

...
...
app = Flask(__name__)
apm = ElasticAPM(app, SERVER_URL='http://DOMAIN:8200', SERVICE_NAME='portfolio', SECRET_TOKEN='*****QB2HMzCL9PgBHVrnxjNXXw5J7bd79DFm6sjBJR5HPXDhcF8MSb3vv4bp***', LOGGING=False)
...
...

The APM server config is:

apm-server:
  host: 0.0.0.0:8200
  secret_token: '*****QB2HMzCL9PgBHVrnxjNXXw5J7bd79DFm6sjBJR5HPXDhcF8MSb3vv4bpg**'

output.elasticsearch:
  hosts: ['elasticsearch:9200']
  username: ${ELASTICSEARCH_USER:elastic}
  password: ${ELASTICSEARCH_PASSWORD:changeme}

setup.kibana:
  host: 'kibana:5601'

setup.dashboards.enabled: true

logging.selectors: ["*"]

logging.to_files: true

logging.files:
  path: /var/log/apm-server

But when i call the flask application from the command line, neither the APM values don't get submitted to APM server nor i can view them in Kibana dashboard.

From APM server logs it is evident that APM parameters reached to the server but due to token error server refused them although token is same in both APM config and Flask app. Following is the logs from APM server about 'HTTP 401: invalid token'

2018-11-17T06:12:34.393Z ERROR [request] beater/common_handlers.go:467 error handling request {"request_id": "92fb84b6-4b87-49ab-a5c0-6be50a41a6c3", "method": "POST", "URL": "/intake/v2/events", "content_length": 1143, "remote_address": "172.24.0.1", "user-agent": "elasticapm-python/4.0.1", "response_code": 401, "error": "invalid token"}

Where could be the issues ?

2

Hello,
Your secret token is exactly the one you pasted in the examples? Because if so, they actually don't match... and also I'd advise to not paste secrets in public forums :slight_smile:
Any case, can you double check that the token is the same, please? Everything else seems fine in your example.
The only reasons for a 401 are a not matching token, or a bug on our side.

3

Hi, I have double checked and found tokens are same in the configurations and the secret token is not the one that i am using(changed).

4

Hi,
Does that happen to you in production only, local only, or both?
Another possibility pointed out by a colleague is if there is a proxy filtering out the auth header. Could that be happening?

If you try it without token, or with a simplistic token just for testing (like "abc"), does it work?

5

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.