# Secure Communications from Beats to Elasticsearch **URL:** \ **Category:** Elasticsearch\ **Tags:** elastic-stack-security\ **Created:** [November 16, 2020, 1:31am UTC](https://discuss.elastic.co/t/secure-communications-from-beats-to-elasticsearch/255460 "2020-11-16T01:31:00Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![inf](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/inf/32/85793_2.png) [@inf](https://discuss.elastic.co/u/inf)\ **Post date:** [November 16, 2020, 1:31am UTC](https://discuss.elastic.co/t/secure-communications-from-beats-to-elasticsearch/255460/1 "2020-11-16T01:31:00Z")
Using Filebeat's article on ... found [here](https://www.elastic.co/guide/en/beats/filebeat/current/securing-communication-elasticsearch.html) as reference, we see that it says that "Filebeat can use any of the following authentication methods". 3 authentication methods were then listed. My question is: are we only able to use PKI certificates as the sole authentication method? I was examining an existing setup done by another colleague (who left), who had both PKI certificate AND token-based API authentication in the Beats yml file. When I commented out one or another and restarted the Beats, it no longer works. I created a brand new setup altogether and tried using only PKI certificates as authentication and got the error "missing authentication credentials for REST request". ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [November 16, 2020, 1:49am UTC](https://discuss.elastic.co/t/secure-communications-from-beats-to-elasticsearch/255460/2 "2020-11-16T01:49:37Z")
> [@inf](#): > > When I commented out one or another and restarted the Beats, it no longer works. Sharing the config and errors you saw would be helpful. ---
**Author:** ![inf](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/inf/32/85793_2.png) [@inf](https://discuss.elastic.co/u/inf)\ **Post date:** [November 16, 2020, 1:52am UTC](https://discuss.elastic.co/t/secure-communications-from-beats-to-elasticsearch/255460/3 "2020-11-16T01:52:54Z")
Sorry for the lack of details, here goes: This is a snippet when I ran `.\winlogbeat -e`: ```auto 2020-11-16T09:15:15.195+0800 ERROR pipeline/output.go:100 Failed to connect to backoff(elasticsearch(https://[IP address]:9200)): 401 Unauthorized: {"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":["Basic realm=\"security\" charset=\"UTF-8\"","Bearer realm=\"security\"","ApiKey"]}}],"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":["Basic realm=\"security\" charset=\"UTF-8\"","Bearer realm=\"security\"","ApiKey"]}},"status":401} ``` This is the relevant part in `winlogbeat.yml`: ```auto output.elasticsearch: # Array of hosts to connect to. hosts: ["[IP address]:9200"] # Protocol - either `http` (default) or `https`. protocol: "https" ssl.certficate_authorities: ['C:\Program Files\Elastic\winlogbeat\ca.crt'] ssl.certificate: C:\Program Files\Elastic\winlogbeat\master-node.crt ssl.key: C:\Program Files\Elastic\winlogbeat\master-node.key ssl.verification_mode: none ``` When I added a valid `username` and `password`, or valid `api_key`, it works fine. ---
**Author:** ![TimV](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/timv/32/13162_2.png) [@TimV](https://discuss.elastic.co/u/TimV)\ **Post date:** [November 16, 2020, 4:27am UTC](https://discuss.elastic.co/t/secure-communications-from-beats-to-elasticsearch/255460/4 "2020-11-16T04:27:27Z")
This core of the problem here is terminology - Elasticsearch use specific terms with specific meanings here, but it's hard to capture that nuance in a summary document like the one you refer to. Your use of "PKI" in the description of what you inherited, is not quite accurate in Elasticsearch terms. In Elasticsearch: - SSL _client authentication_ refers to the use of client-side X.509 certificates as part of SSL/TLS handshaking. If you set the [`xpack.security.http.ssl.client_authentication`](https://www.elastic.co/guide/en/elasticsearch/reference/7.9/security-settings.html#http-tls-ssl-settings) setting to `required` then all connections to the cluster must provide a valid, trusted client certificate. This is what is commonly referred to as "mutual TLS" (sometimes mTLS). By itself, that is a network based control only. It does not integrate with Elasticsearch's internal RBAC security model. - [_PKI authentication_](https://www.elastic.co/guide/en/elasticsearch/reference/7.9/pki-realm.html) refers to using client-side X.509 certificates as a mechanism to authenticate Elasticsearch users. It is an alternative to providing a username & password, or an API key for authentication. Within that Elasticsearch terminology, the use of client certificates for TLS is _not_, by itself, _"PKI"_, it is "mutual TLS" or "SSL client authentication". You can use those certificates to implement PKI authentication, but they are distinct terms & concepts. > I was examining an existing setup done by another colleague (who left), who had both PKI certificate AND token-based API authentication in the Beats yml file Strictly speaking, from a Beat yml file, you can only tell that it provided client certificates, you cannot tell whether they are used for "PKI" without looking at the Elasticsearch configuration. The `elasticsearch.yml` will determine whether SSL client certificates can be used for PKI based integration into the RBAC model, or are merely used as a network control. > [@inf](#): > > I created a brand new setup altogether and tried using only PKI certificates as authentication and got the error "missing authentication credentials for REST request". Assuming that you used the same client certificate & key that works for the existing beats setup, this implies that the Elasticsearch server is set up for _SSL client authentication_ but not PKI. However there are other explanations, so if you have access to the `elasticsearch.yml` config, that will help narrow it down. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [December 14, 2020, 4:27am UTC](https://discuss.elastic.co/t/secure-communications-from-beats-to-elasticsearch/255460/5 "2020-12-14T04:27:41Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.