How can we ensure Data Encrypted?

Elastic Stack Elasticsearch
1

Hello, I have installed Shield in Elasticsearch, data pushing through logstash. How can we ensure the data is encrypted in Elasticsearch. could you please give the steps to check.. all possible ways..

2

Shield can encrypt data transferred between the client and the cluster as well as between the nodes in the cluster. It does however not support encryption on disk. Which type of encryption are you referring to?

3

Sorry, I have not mentioned in my previous message, I have installed Shield as Elasticsearch plugin and enabled shield with Organization given certificate. The certificate I have imported to .JKS file with keytool and configured same to Elasticsearch config file i.e elasticsearch.yml file.. while starting elasticsearch server instance it asking for password because.. i have updated the following fields like below
shield.ssl.keystore.password: ${prompt.secret}
shield.ssl.keystore.key_password: ${prompt.secret}

Thing is it asking twice each password wile start, what could be the reason ?
My first question is how we can ensure data is encrypted...? please suggest me possible ways to verify the data is encrypted..

I have Organization certificate so, I have imported that to keystore and configured in elastic search i am seeing below exception
[2016-02-22 16:01:17,023][WARN ][shield.transport.netty ] exception caught on transport layer [[id: 0xd19d3e02, /0:0:0:0:0:0:0:1:54172 => /0:0:0:
0:0:0:0:1:9300]], closing connection
javax.net.ssl.SSLHandshakeException: no cipher suites in common

please help me resolve

4

This is a bug in elasticsearch. For more information see ${prompt.text} and ${prompt.secret} double prompting · Issue #11564 · elastic/elasticsearch · GitHub

The exception usually comes from a misconfiguration of certificates and keystores. You'll need to list all of the steps you took to help us diagnose the exception.

In terms of verifying that data is encrypted, right now it is in use because the handshakes are failing. You won't see exceptions when configured properly. If you do not trust the shield configuration is requiring encryptiong, you could:

  1. Try to connect to the cluster with a node that has SSL disabled and verify it cannot connect
  2. Sniff the network traffic with wireshark/tcpdump and validate that it is not in plain text
5

Could you please help me with minimum configurations.. for Communication encryption between nodes and http

I have done the following steps to configure certificate
step1: As per https://www.elastic.co/guide/en/shield/current/certificate-authority.html
executed below statements

  • mkdir -p ca\private ca\certs ca\conf
  • cd ca
  • echo '01' > serial
  • echo $null > index.txt
    Step2: copied the config template and saved in the file conf\caconfig.cnf and then Add the full path to this newly created CA then executed below command
    step3: openssl req -new -x509 -extensions v3_ca -keyout private\cakey.pem -out certs\cacert.pem -days 1460 -config conf\caconfig.cnf
    -- entered: test123 password
    As per link https://www.elastic.co/guide/en/shield/current/ssl-tls.html
    step4: keytool -importcert -keystore devNode.jks -file certs\cacert.pem -alias my_ca
    step5: keytool -genkey -alias devNode -keystore devNode.jks -keyalg RSA -keysize 2048 -validity 712 -ext san=dns:localhost,ip:127.0.0.1
    --- given some test data when Keytool prompts for keytool -genkey,
    step6: keytool -certreq -alias devNode -keystore devNode.jks -file devNode.csr -keyalg rsa -ext san=dns:localhost,ip:127.0.0.1
    step6: openssl ca -in devNode.csr -notext -out devNode-signed.crt -config conf\caconfig.cnf -extensions v3_req

i am seeing below message for the above command
Using configuration from conf\caconfig.cnf
Loading 'screen' into random state - done
Enter pass phrase for C:/elk/ca/private/cakey.pem:
wrong number of fields on line 1 (looking for field 6, got 1, '' left)
unable to write 'random state'

so not able to import the signed certificate to .JKS...
step7: keytool -importcert -keystore devNode.jks -file devNode-signed.crt -alias devNode
even the generate the signed certificate also not able to import and enable the certificate

I have tried with Organization given signed certificate, I am able to imported to .JKS but I am seeing below error while start elasticsearch
[2016-02-23 17:28:01,378][INFO ][shield.transport ] [Asylum] publish_address {127.0.0.1:9300}, bound_addresses {
[::1]:9300}, {127.0.0.1:9300}
[2016-02-23 17:28:01,398][INFO ][discovery ] [Asylum] elasticsearch/6MMUh1e3TEe15fccyu9O8A
[2016-02-23 17:28:01,908][WARN ][shield.transport.netty ] [Asylum] exception caught on transport layer [[id: 0xbab3fc6
b, /0:0:0:0:0:0:0:1:53251 => /0:0:0:0:0:0:0:1:9300]], closing connection
javax.net.ssl.SSLHandshakeException: no cipher suites in common
at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290)
at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513)
at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:790)
at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:758)
at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624)
at org.jboss.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1218)
at org.jboss.netty.handler.ssl.SslHandler.decode(SslHandler.java:852)
at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:425)
at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeli
ne.java:791)

I have tried with below steps also but I am seeing same above error
keytool -genkey -alias mykey -keyalg RSA -keysize 2048 -sigalg SHA256withRSA -validity 712 -keypass privatepassw0rd -keystore node01.jks -storepass passw0rd
keytool -export -alias mykey -file server.cer -keystore node01.jks -storepass passw0rd
keytool -import -alias mykey -file server.cer -keystore trustnode.jks -storepass passw0rd
keytool -list -v -keystore node01.jks -storepass passw0rd

elasticsearch.yml

shield.ssl.keystore.path: D:/elk/elasticsearch-2.2.0/config/shield/node01.jks
shield.ssl.keystore.password: passw0rd
shield.ssl.keystore.key_password: privatepassw0rd
shield.transport.ssl: true
shield.http.ssl: true
please help me where i am missing?

6

Lets start with trying to determine the error with the signing of the CSR that our instructions mention. Our documentation says to execute touch index.txt but you executed echo $null > index.txt. Can you please delete that file and create a completely empty file using the touch command? The error you got is because of a malformed index.txt file.