Hi , Thanks in advance for helping.
I am trying to setup a secure ELK infra with a beats to push logs to it. This ELK infra has 1 Master and 2 data nodes, 1 kibana, 1 logstash and 1 filebeats. This Elastic cluster is secure for which following are the configurations:
Master:
bootstrap.memory_lock: false
cluster.name: ClusterI4
http.port: 9200
network.host: 172.20.5.213
node.data: false
node.ingest: true
node.master: true
node.name: MasterNode
path.data: S:\Elastic\Data
path.logs: S:\Elastic\Logs
transport.tcp.port: 9300
xpack.license.self_generated.type: basic
xpack.security.enabled: true
discovery.zen.ping.unicast.hosts: ["172.20.5.213","172.20.5.214:9301"]
xpack.monitoring.collection.enabled: true
xpack.security.transport.ssl.enabled: true
discovery.zen.minimum_master_nodes: 2
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: S:\Elastic\Config\certificate-bundle\elastic-stack-ca.p12
xpack.security.transport.ssl.truststore.path: S:\Elastic\Config\certificate-bundle\elastic-stack-ca.p12
Data:
bootstrap.memory_lock: true
cluster.name: ClusterI4
http.port: 9200
network.host: 172.20.5.214
node.data: true
node.ingest: false
node.master: false
node.name: Data1
path.data: S:\Elastic\Elasticsearch\data
path.logs: S:\Elastic\Elasticsearch\logs
transport.tcp.port: 9301
xpack.license.self_generated.type: basic
xpack.security.enabled: true
discovery.zen.ping.unicast.hosts: ["172.20.5.213:9300","172.20.5.214:9301"]
xpack.monitoring.collection.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: S:\Elastic\Elasticsearch\config\certificate-bundle\elastic-stack-ca.p12
xpack.security.transport.ssl.truststore.path: S:\Elastic\Elasticsearch\config\certificate-bundle\elastic-stack-ca.p12
The whole cluster is properly connected and working fine but the data is not able to flow from filebeats to logstash to elastic cluster. I guess the reason is the ssl configuration in both.
Logstash:
> input
> {
> beats{
> port => 5044
> ssl => true
> ssl_certificate_authorities =>["C:\filebeat-7.1.0-windows-x86_64\filebeat-7.1.0-windows-x86_64\ca\ca.crt"]
> ssl_certificate => "C:\filebeat-7.1.0-windows-x86_64\filebeat-7.1.0-windows-x86_64\ca\ca.crt"
> ssl_key => "C:\filebeat-7.1.0-windows-x86_64\filebeat-7.1.0-windows-x86_64\ca\ca.key"
> ssl_verify_mode => "force_peer"
> }
>
> }
>
> filter {
>
> }
> output {
> elasticsearch {
> hosts => ["172.20.5.213:9200"]
> index => "%{indice}-%{+xxxx.ww}"
> ssl => true
> keystore => "C:\logstash-6.5.4\logstash-6.5.4\config\certificate-bundle\elastic-stack-ca.p12"
> truststore => "C:\logstash-6.5.4\logstash-6.5.4\config\certificate-bundle\elastic-stack-ca.p12"
> }
> }
Filebeat:
filebeat.inputs: - type: log enabled: true paths: - C:\logs\*.nlog output.logstash: hosts: ["localhost:5044"] ssl.certificate_authorities: ["C:\filebeat-7.1.0-windows-x86_64\filebeat-7.1.0-windows-x86_64\ca\ca.crt"] ssl.certificate: "C://filebeat-7.1.0-windows-x86_64//filebeat-7.1.0-windows-x86_64//ca//ca.crt" ssl.key: "C://filebeat-7.1.0-windows-x86_64//filebeat-7.1.0-windows-x86_64//ca//ca.crt"
Logs in filebeat were as follows:
2019-05-29T07:22:35.631-0700 ERROR instance/beat.go:802 Exiting: error initializing publisher: 1 error: tls: found a certificate rather than a key in the PEM for the private key
Can someone help me to rectify the log flow.
Thanks