Logstash to elastic output SSL

Elastic Stack Logstash
1

I have configured SSL for all ELASTIC cluster nodes
Blowed if I can find any reliable information regarding how to subsequently configure the logstash output plugin once I restart my ES cluster with ssl enabled.

I 'think' I ONLY need the elastic cluster ca.crt and the following lines in the output plugin
ssl => true
cacert => /path/to/ca.crt

Maybe I should have included the logstash nodes in my gencert run and a logstash server cert and key is required somewhere too?

2

What did you configure in elasticsearch.yml ?

There's a number of different possible configurations for SSL in elasticsearch, and the appropriate logstash configuration will be dependent on what changes you made.

3

I had to import the ca cert to the Java keystore. try that.
keytool -import -alias alias -keystore path-to-jre/lib/security/cacerts -file path-to-certificate-file

4

my es clients elasticsearch.yml : after following
https://www.elastic.co/guide/en/x-pack/5.4/ssl-tls.html#installing-node-certificatescluster.name: cluster.name:
node.name: <hostname>
path.data: </path/to/logs>
path.logs: </path/to/logs>
network.host: eth0
discovery.zen.ping.unicast.hosts: [ "n.n.n.n", "n.n.n.n", "n.n.n.n" ]
http.port: 9200
bootstrap.memory_lock: true
node.master: false
node.data: false
node.ingest: true
thread_pool.bulk.size: 3
thread_pool.bulk.queue_size: 10000
thread_pool.index.size: 3
thread_pool.index.queue_size: 10000
indices.fielddata.cache.size: 30%

xpack.ssl.certificate_authorities: /etc/<path/to/ca.crt
xpack.ssl.key: /etc/<path/to/hostname.key>
xpack.ssl.certificate: </path/to/hostname.crt>
xpack.security.transport.ssl.enabled: false
xpack.security.http.ssl.enabled: false
xpack.security.enabled: true
action.auto_create_index: true
xpack.security.audit.enabled: true

5

I'm on 5.4 - I didn't use java keystore - I followed the guide at
https://www.elastic.co/guide/en/x-pack/5.4/ssl-tls.html#installing-node-certificates

6

You've disabled SSL on both the transport port and the http port, so you're not actually using SSL at all.

If you want Logstash to use SSL you'll need to enable SSL on the http port.

xpack.security.http.ssl.enabled: true

Then, what you initially proposed for the Logstash config should be correct, provided that the IP/hostname you entered into certgen match your ES node's address.

7

Yes I purposely set ssl disabled for the moment whilst working on the Logstash config.

The ca cert generated (on one of the es nodes) by certgen created keys and certs for all my es nodes and Logstash nodes plus the ca cert and key

It’s the ca.crt required on the Logstash node? Not it’s own cert?

 Sent from my iPhone

8

Yes, I assumed that was the case, but I wanted to be really clear.

It depends on exactly what you're trying to achieve, but yes, it's the CA cert.

SSL can be used for 3 things (but doesn't need to do all of them)

  1. Confidentiality (Encryption)
  2. Server identity
  3. Client identity

Most of the time only (1) and (2) apply.
In a web browser environment, SSL makes sure your connection is encrypted, and the certificate checking makes sure you're connecting to the right server, but it's not normally used to check the identity of the user operating the browser. It can be, but normally you just pass a username + password over the encrypted SSL connection.

Logstash is the same. It is the client to the ES server, and you can use a client certificate to establish the identity of the logstash process, but it is more commonly that case that use just use a username + password.
That means your logstash process doesn't need its own certificate because it's not trying to establish its own identity via SSL certs. It does, however, need to know how to check the identity of the server. For that it needs a copy of the CA certificate. The CA is the participant that is asserting the identity of the ES server, and the Logstash process needs to trust that CA by being configured with a copy of the CA cert.

If you to use SSL to eastblish the Logstash process's identity, then you'll need a different configuration on both the Logstash side and the ES side.

9

So at this stage I only want to encrypt the data in transit between the logstash and the ES
so I have enabled SSL on the ES cluster and restarted those nodes
and added
ssl => true
cacert => /path/to/ca.crt
on the logstash nodes output filter
I have added nothing to the logstash.yml

Now the service starts but no traffic is output
logstash.outputs.elasticsearch Attempted to resurrect connection to dead ES instance but got error
Got response code '401' contacting Elasticsearch at URL 'https://n.n.n.n:9200/

10

That looks like a consequence of turning on X-Pack security rather than SSL.
Sorry, I had assumed you already had the security side working.

You need to add user and password to your logstash config.

There's a guide for Logstash + Elasticsearch X-Pack Security that you might find helpful.

11

Authentication is/was already working - already have that configured following aforementioned guide.

I figured out the syntax required anyway eventually and will post here tomorrow

12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.