# How to Setup FileBeat with Basic Auth for LogStash Output? **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [December 11, 2015, 4:49am UTC](https://discuss.elastic.co/t/how-to-setup-filebeat-with-basic-auth-for-logstash-output/36937 "2015-12-11T04:49:12Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![kpcool](https://avatars.discourse-cdn.com/v4/letter/k/90db22/32.png) [@kpcool](https://discuss.elastic.co/u/kpcool)\ **Post date:** [December 11, 2015, 4:49am UTC](https://discuss.elastic.co/t/how-to-setup-filebeat-with-basic-auth-for-logstash-output/36937/1 "2015-12-11T04:49:13Z")
I plan to use FileBeat for log monitoring and push out the logs to logstash on a central server, which has http basic auth setup, to prevent unauthorized inputs. Without auth, the setup works fine, no issues. However, when I enable the basic auth on central server with logstash, I get > 2015/12/11 04:41:14.918593 single.go:75: INFO Error publishing events (retrying): lumberjack protocol error I have put 'username' and 'password' in the output section of 'logstash', but seems like it is not getting picked up properly. (Output section of filebeat.yml) > output: > logstash: > # The Logstash hosts > hosts: ["127.0.0.1:5047"] > index: jsonfilebeat > username: "uname" > password: "pwd" 5047 is then proxied to 5046 which Logstash is listening via Beats Input (input section of logstash conf). > input { > beats { > port =\> 5046 > type =\> "json" > } > } Where is that I am going wrong? Or is this setup not supported? Please help clarify. Thanks, ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [December 11, 2015, 7:16am UTC](https://discuss.elastic.co/t/how-to-setup-filebeat-with-basic-auth-for-logstash-output/36937/2 "2015-12-11T07:16:35Z")
The Lumberjack/Beats protocol isn't HTTP based so directing the traffic at an HTTP proxy that requires authentication just won't work. The Logstash output doesn't have any `username` and `password` options. ---
**Author:** ![kpcool](https://avatars.discourse-cdn.com/v4/letter/k/90db22/32.png) [@kpcool](https://discuss.elastic.co/u/kpcool)\ **Post date:** [December 11, 2015, 10:10am UTC](https://discuss.elastic.co/t/how-to-setup-filebeat-with-basic-auth-for-logstash-output/36937/3 "2015-12-11T10:10:25Z")
Is there a way that I can secure the logstash beats input, so only authorized users would be able to post to it? I couldn't find a way to secure it via Shield either. ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [December 11, 2015, 3:45pm UTC](https://discuss.elastic.co/t/how-to-setup-filebeat-with-basic-auth-for-logstash-output/36937/4 "2015-12-11T15:45:42Z")
Logstash and filebeat use certificates. filebeat already support client certificates (TLS client authentication), but this is still required to be implemented in [logstash](https://github.com/logstash-plugins/logstash-input-beats/issues/8). As workaround for now you can try to tunnel the beats connection via vpn, ssh or [stunnel](https://www.stunnel.org/auth.html). When using a tunnel configure your logstash to listen on localhost only (port will not be available from outside world): ``` input { beats { host => "127.0.0.1", port => 5044 } } ``` ---
**Author:** ![kpcool](https://avatars.discourse-cdn.com/v4/letter/k/90db22/32.png) [@kpcool](https://discuss.elastic.co/u/kpcool)\ **Post date:** [December 11, 2015, 4:53pm UTC](https://discuss.elastic.co/t/how-to-setup-filebeat-with-basic-auth-for-logstash-output/36937/5 "2015-12-11T16:53:47Z")
Just for confirmation, will setting up the certificates in logstash and filebeat as mentioned in [https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-elk-stack-on-ubuntu-14-04](https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-elk-stack-on-ubuntu-14-04) serve the purpose of authentication? If it does then it would be good to add this somewhere in reference documentation so people don't have to dig around for information on internet. ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [December 11, 2015, 5:29pm UTC](https://discuss.elastic.co/t/how-to-setup-filebeat-with-basic-auth-for-logstash-output/36937/6 "2015-12-11T17:29:07Z")
You've been asking for mutual authentication, which is not yet supported in logstash (see my other comment). No, for your use-case it's not enough. The tutorial only sets up a server certificate. That is, filebeat with check the server being a valid sink for sending events too. But logstash CAN NOT check filebeat being allowed to send and will still accept every beats connection (see [github issue](https://github.com/logstash-plugins/logstash-input-beats/issues/8) ). Having the github issue resolved, mutual auth is possible. Setting up CAs and doing all the certificates work (sign, revoke, renew, distribute ...) is a completely different story. The setup described in the tutorial is the most 'simplest' using self-signed certificate to get you somehow started, but in bigger setups You'd prefer having CAs for manageability. ---
**Author:** ![kpcool](https://avatars.discourse-cdn.com/v4/letter/k/90db22/32.png) [@kpcool](https://discuss.elastic.co/u/kpcool)\ **Post date:** [December 12, 2015, 4:32am UTC](https://discuss.elastic.co/t/how-to-setup-filebeat-with-basic-auth-for-logstash-output/36937/7 "2015-12-12T04:32:40Z")
> [@steffens](#): > > but in bigger setups You'd prefer having CAs for manageability I agree that this solution will not be a mutual authentication but I presume this is a safe workaround till the mutual authentication is resolved. I understand that having a CA sign the cert and using it will aid on bigger setups. But for smaller setup (single node logstash and multiple beat publishers), if private/public cert are not shared anywhere else, then this would be secure enough and prevent anyone from dumping data in logstash. If I understand it properly, then logstash will reject any messages which are not signed with the assigned public certificate. Is this not a correct understanding? ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [December 14, 2015, 12:57am UTC](https://discuss.elastic.co/t/how-to-setup-filebeat-with-basic-auth-for-logstash-output/36937/8 "2015-12-14T00:57:05Z")
> [@kpcool](#): > > If I understand it properly, then logstash will reject any messages which are not signed with the assigned public certificate. Is this not a correct understanding? No. Messages are not signed. The connection filebeat-\>logstash is secured using TLS (same technology used for https). Messages are not signed (plus for signing you'd need a private key and verify using a public key) at all. Client certificates MUST be used with TLS if you don't want all possible clients to send data to logstash. See "Client-authenticated TLS handshake" on [TLS wikipedia article](https://en.wikipedia.org/wiki/Transport_Layer_Security). For this to work, servers must be configured to require client authentication in first place. This is currently not possible with logstash. ---
**Author:** ![lookingcloudy](https://avatars.discourse-cdn.com/v4/letter/l/43a26b/32.png) [@lookingcloudy](https://discuss.elastic.co/u/lookingcloudy)\ **Post date:** [December 30, 2015, 5:04pm UTC](https://discuss.elastic.co/t/how-to-setup-filebeat-with-basic-auth-for-logstash-output/36937/9 "2015-12-30T17:04:40Z")
Am I reading this right? Filebeats currently has no support for security? Surely that can't be the case. ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [December 30, 2015, 5:11pm UTC](https://discuss.elastic.co/t/how-to-setup-filebeat-with-basic-auth-for-logstash-output/36937/10 "2015-12-30T17:11:27Z")
filebeat has same support as logstash-forwarder used to have, plus some more fine-grained TLS configs (e.g. choose TLS version or configure ciphers). Connection can encrypted via TLS + server certificated is validated. Filebeat itself supports TLS client-auth, BUT logstash must enforce (ask for certificate) client authentication, which is not implemented yet (see github issue). ---
**Author:** ![CptPk](https://avatars.discourse-cdn.com/v4/letter/c/6bbea6/32.png) [@CptPk](https://discuss.elastic.co/u/CptPk)\ **Post date:** [February 5, 2016, 10:11pm UTC](https://discuss.elastic.co/t/how-to-setup-filebeat-with-basic-auth-for-logstash-output/36937/11 "2016-02-05T22:11:02Z")
i am a bit confused does that really mean that its impossible to ship logs to logstash =\> elasticsearch in a secure way without VPN / stunnel? Thats because logstash does not force client certificates? Does anybody know a alternative? ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [February 8, 2016, 2:45pm UTC](https://discuss.elastic.co/t/how-to-setup-filebeat-with-basic-auth-for-logstash-output/36937/12 "2016-02-08T14:45:44Z")
In TLS certificates are normally provided by the Server (logstash in this case) to the client and the client has to validate the certificate. Once handshake and certificate validation succeed, communication will be encrypted. It's the same technology used in https for example. This part is support in beats and logstash. There exists a second optional mode in TLS the server asking the client for the client certificate. If server asks for client-certificate, the client will send it's own certificate to the server and the server has to validate the certificate. Since client-auth is optional, it has no effect on the connection being encrypted. This mode is not support in logstash server. ---
**Author:** ![CptPk](https://avatars.discourse-cdn.com/v4/letter/c/6bbea6/32.png) [@CptPk](https://discuss.elastic.co/u/CptPk)\ **Post date:** [February 8, 2016, 3:02pm UTC](https://discuss.elastic.co/t/how-to-setup-filebeat-with-basic-auth-for-logstash-output/36937/13 "2016-02-08T15:02:55Z")
Thank you very much for clarify that. But what i mean with secure way is that if i want to make my logstash instance rechable from outside ANY client could post data to my logstash instance. www =\> my logstash server =\> ES www EVIL client =\> my logstash server =\> ES What i would expect is any way like: www EVIL client --- Username / Password / LDAP Auth / Client certificate =\> my logstash server =\> ES ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [February 9, 2016, 12:52am UTC](https://discuss.elastic.co/t/how-to-setup-filebeat-with-basic-auth-for-logstash-output/36937/14 "2016-02-09T00:52:59Z")
Yes, this might indeed a problem. You can [follow progress on the related ticket on github](https://github.com/logstash-plugins/logstash-input-beats/issues/8). ---
**Author:** ![CptPk](https://avatars.discourse-cdn.com/v4/letter/c/6bbea6/32.png) [@CptPk](https://discuss.elastic.co/u/CptPk)\ **Post date:** [February 9, 2016, 7:00am UTC](https://discuss.elastic.co/t/how-to-setup-filebeat-with-basic-auth-for-logstash-output/36937/15 "2016-02-09T07:00:14Z")
Thanks for clarification Steffen ---
**Author:** ![tuananhtd](https://avatars.discourse-cdn.com/v4/letter/t/b5ac83/32.png) [@tuananhtd](https://discuss.elastic.co/u/tuananhtd)\ **Post date:** [March 30, 2016, 10:44am UTC](https://discuss.elastic.co/t/how-to-setup-filebeat-with-basic-auth-for-logstash-output/36937/16 "2016-03-30T10:44:06Z")
Now that the mentioned [Github issue](https://github.com/logstash-plugins/logstash-input-beats/issues/8) is closed. Has anyone give it a try? It seem like with self-singed certificate we have to copy the certificate and the private key to the filebeat server? Also we must use the certificate as the CA. I do not know much about SSL/TLS protocol could any one give some thoughts on this? Thanks! ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [March 30, 2016, 12:01pm UTC](https://discuss.elastic.co/t/how-to-setup-filebeat-with-basic-auth-for-logstash-output/36937/17 "2016-03-30T12:01:21Z")
I didn't test this yet (there might be some subtle errors in examples). You will at least need most recent logstash-input-beats plugin (hopefully including the PR already) to configure client auth. For client-auth you need to have some certificate authority (CA) to verify the clients certificate + enable authentication server side (as Server must request client for its certificate). Set list of CAs in `ssl_certificate_authorities` and enable server verification by setting `ssl_verify_mode` to `"force_peer"` (most strict option). List of CAs must either include the self-signed certificate used by the client or better the root certificate used to sign the client certificate. Advantage of root certificate is, you have to configure logstash only once and can create/revoke client certificate any time (add more clients to network without updating logstash config). On client side we need the actual certificate + key. In filebeat configure `certificate` and `certificate_key`. For testing the config you can start with having logstash and filebeat running on same host (with same domain) and use same certificate for logstash and filebeat. But in production you will need separate certificates for both, logstash and filebeat. Following [this tutorial](https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-elk-stack-on-ubuntu-14-04) (assuming your are using the domain names + self-signed certificates without root CAs) check out the "Generate SSL Certificates" section and adapt: ```auto $ sudo mkdir -p /etc/pki/tls/certs $ sudo mkdir /etc/pki/tls/private $ cd /etc/pki/tls ``` ```auto $ sudo openssl req -subj '/CN=/' -x509 -days 3650 -batch -nodes -newkey rsa:2048 -keyout private/logstash.key -out certs/logstash.crt $ sudo openssl req -subj '/CN=/' -x509 -days 3650 -batch -nodes -newkey rsa:2048 -keyout private/filebeat.key -out certs/filebeat.crt ``` The `02-beats-input.conf` must say: ```auto input { beats { port => 5044 ssl => true # enable TLS/SSL # configure logstash server certificate being presented to filebeat ssl_certificate => "/etc/pki/tls/certs/logstash.crt" ssl_key => "/etc/pki/tls/private/logstash.key" # configure client auth + filebeat cert for validation ssl_verify_mode => "force_peer", ssl_certificate_authorities => ["/etc/pki/tls/certs/filebeat.crt"], } } ``` In filebeat.yml enable TLS + add certificate: ```auto logstash: tls: # List of root certificates for server verifications certificate_authorities: ["/etc/pki/tls/certs/logstash.crt"] # client certificate + key for client auth (if requested by server) certificate: "/etc/pki/tls/certs/filebeat.crt" certificate_key: "/etc/pki/tls/private/filebeat.key" ``` NOTE: when copying certificates + key files only copy files required. Logstash does not require filebeat.key and filebeat does not require logstash.key. ---
**Author:** ![tuananhtd](https://avatars.discourse-cdn.com/v4/letter/t/b5ac83/32.png) [@tuananhtd](https://discuss.elastic.co/u/tuananhtd)\ **Post date:** [March 30, 2016, 1:57pm UTC](https://discuss.elastic.co/t/how-to-setup-filebeat-with-basic-auth-for-logstash-output/36937/18 "2016-03-30T13:57:46Z")
@steffens thank you for the great answer. Thing got much clearer now! I just have one question regarding the protocol. Is that the certificate contains of one party identity information plus the the signature which is that info encoded with some private key? And in the verification process, the signature is decoded with the corresponding public key in this case the CAs if the decoded info match (with the unencoded version) then the certificate is verified? ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [March 30, 2016, 10:34pm UTC](https://discuss.elastic.co/t/how-to-setup-filebeat-with-basic-auth-for-logstash-output/36937/19 "2016-03-30T22:34:55Z")
TLS is just another network protocol sitting in between TCP and the application layer protocol (It's the protocol driving HTTPS for example). Check out [wikipedia](https://en.wikipedia.org/wiki/Transport_Layer_Security#Digital_certificates) + [TLS handshake](https://en.wikipedia.org/wiki/Transport_Layer_Security#Protocol_details). It's right, certificates do use public key cryptography for validation. Plus domain name and CA-chain (unless self-signed) is used to validate a certificate. Ever got a Certificate warning in your browser? This is if a certificate couldn't be validated or doesn't match domain name (or IP). The option `-keyout` in the openssl command writes the private key, that is `filebeat.key` and `logstash.key` are the private keys, which should not be shared. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 9:54pm UTC](https://discuss.elastic.co/t/how-to-setup-filebeat-with-basic-auth-for-logstash-output/36937/20 "2017-07-05T21:54:02Z")