Prevent inclusion of Authorization header

Elastic Stack Beats
1

To get FileBeat working with AWS ES, the Authorization http header must not be included in the http request.

How can I prevent FileBeat sending a basic auth header?

Setting -E output.elasticsearch.username="" -E output.elasticsearch.password="" in the docker command does not work.

Example Docker Command

    docker run \
        docker.elastic.co/beats/filebeat-oss:6.5.4 \
        setup -E setup.kibana.host="https://*.eu-west-1.es.amazonaws.com:443" \
        -E output.elasticsearch.hosts=["https://*.eu-west-1.es.amazonaws.com:443"]

Output from FileBeat:

Exiting: Couldn't connect to any of the configured Elasticsearch hosts. Errors: [Error connection to Elasticsearch https://*.eu-west-1.es.amazonaws.com:443: 403 Forbidden: {"message":"Authorization header requires 'Credential' parameter. Authorization header requires 'Signature' parameter. Authorization header requires 'SignedHeaders' parameter. Authorization header requires existence of either a 'X-Amz-Date' or a 'Date' header. Authorization=Basic ZWxhc3RpYzpjaGFuZ2VtZQ=="}]

Another user also had the same issue, but didn't ask about the Authorization header.

2

The error message seems to suggest that AWS ES requires you to have an Authorization header in a special format include credentials and signature. Beats are not tested with AWS IAM, and I don't think that Beats support AWS IAM functionality.

3

AWS ES requires credentials and signature only if an Authorization header is present. If the header is not present, requests do not need to be signed.

I have tested removal the Authorization header with an nginx container proxying requests, but this is quite a bit of additional complexity for deployment.

1 Like
4

Hm... I don't think filebeat will send create an Authorization header if no username or password is configured. You can try to add -E 'output.elasticsearch={user: "", password: ""} to overwrite the settings with empty strings in the containers config file.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.