Enabling beats to logstash flow for a secure elastic cluster

Hi , Thanks in advance for helping.
I am trying to setup a secure ELK infra with a beats to push logs to it. This ELK infra has 1 Master and 2 data nodes, 1 kibana, 1 logstash and 1 filebeats. This Elastic cluster is secure for which following are the configurations:
Master:

bootstrap.memory_lock: false
cluster.name: ClusterI4
http.port: 9200
network.host: 172.20.5.213
node.data: false
node.ingest: true
node.master: true
node.name: MasterNode
path.data: S:\Elastic\Data
path.logs: S:\Elastic\Logs
transport.tcp.port: 9300
xpack.license.self_generated.type: basic
xpack.security.enabled: true
discovery.zen.ping.unicast.hosts: ["172.20.5.213","172.20.5.214:9301"]
xpack.monitoring.collection.enabled: true
xpack.security.transport.ssl.enabled: true
discovery.zen.minimum_master_nodes: 2
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: S:\Elastic\Config\certificate-bundle\elastic-stack-ca.p12
xpack.security.transport.ssl.truststore.path: S:\Elastic\Config\certificate-bundle\elastic-stack-ca.p12

Data:

bootstrap.memory_lock: true
cluster.name: ClusterI4
http.port: 9200
network.host: 172.20.5.214
node.data: true
node.ingest: false
node.master: false
node.name: Data1
path.data: S:\Elastic\ElasticSearch\data
path.logs: S:\Elastic\ElasticSearch\logs
transport.tcp.port: 9301
xpack.license.self_generated.type: basic
xpack.security.enabled: true
discovery.zen.ping.unicast.hosts: ["172.20.5.213:9300","172.20.5.214:9301"]
xpack.monitoring.collection.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: S:\Elastic\ElasticSearch\config\certificate-bundle\elastic-stack-ca.p12
xpack.security.transport.ssl.truststore.path: S:\Elastic\ElasticSearch\config\certificate-bundle\elastic-stack-ca.p12

The whole cluster is properly connected and working fine but the data is not able to flow from filebeats to logstash to elastic cluster. I guess the reason is the ssl configuration in both.

Logstash:

> input 
> {
> 	beats{
> 		port => 5044
> 		ssl => true
> 		ssl_certificate_authorities =>["C:\filebeat-7.1.0-windows-x86_64\filebeat-7.1.0-windows-x86_64\ca\ca.crt"]
> 		ssl_certificate => "C:\filebeat-7.1.0-windows-x86_64\filebeat-7.1.0-windows-x86_64\ca\ca.crt"
> 		ssl_key => "C:\filebeat-7.1.0-windows-x86_64\filebeat-7.1.0-windows-x86_64\ca\ca.key"
> 		ssl_verify_mode => "force_peer"
> 	}
>    
> }
> 
> filter {
> 		
> }
> output {
>   elasticsearch {
>     hosts => ["172.20.5.213:9200"]
> 	index => "%{indice}-%{+xxxx.ww}"
> 	ssl => true
> 	keystore => "C:\logstash-6.5.4\logstash-6.5.4\config\certificate-bundle\elastic-stack-ca.p12"
> 	truststore => "C:\logstash-6.5.4\logstash-6.5.4\config\certificate-bundle\elastic-stack-ca.p12"
>   }
> }

Filebeat:

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - C:\logs\*.nlog
output.logstash:
  hosts: ["localhost:5044"]
  ssl.certificate_authorities: ["C:\filebeat-7.1.0-windows-x86_64\filebeat-7.1.0-windows-x86_64\ca\ca.crt"]
  ssl.certificate: "C://filebeat-7.1.0-windows-x86_64//filebeat-7.1.0-windows-x86_64//ca//ca.crt"
  ssl.key: "C://filebeat-7.1.0-windows-x86_64//filebeat-7.1.0-windows-x86_64//ca//ca.crt"

Logs in filebeat were as follows:
2019-05-29T07:22:35.631-0700 ERROR instance/beat.go:802 Exiting: error initializing publisher: 1 error: tls: found a certificate rather than a key in the PEM for the private key

Can someone help me to rectify the log flow.

Thanks

Your configuration has the same value for output.logstash / ssl.key as it has for output.logstash / ssl.certificate. It wants a key file, not a crt file. You can use openssl to extract the key from a combined key/cert file.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.