I implemented and successfully tested java JEST client as follows
pom.xml
<dependency>
<groupId>io.searchbox</groupId>
<artifactId>jest</artifactId>
<version>6.3.1</version>
</dependency>
<dependency>
<groupId>io.searchbox</groupId>
<artifactId>jest-common</artifactId>
<version>6.3.1</version>
</dependency>
Source code of project JestToSecurES in Eclipse (Neon)
import java.io.InputStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.security.KeyStore;
import java.security.cert.CertificateFacrory;
import java.util.HashMap;
import java.util.Map;
import javax.net.ssl.HostnameVerifier;
import org.apache.http.conn.ssl.NoopHostnameVerifier;
import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
import org.apache.http.nio.conn.SchemeIOSessionStrategy;
import org.apache.http.nio.conn.SSLIOSessionStrategy;
import org.apache.http.ssl.SSLContextBuilder;
import org.apache.http.ssl.SSLContexts;
import com.google.gson.JsonObject;
import io.searchbox.client.JestClient;
import io.searchbox.client.JestClientFactory;
import io.searchbox.client.JestResult;
import io.searchbox.client.config.HttpClientConfig;
import io.searchbox.core.Search;
public class JestConnSecurES {
public static void main() throws Exception {
System.out.println("Running main class JestConnSecurES of project JestToSecurES);
System.out.println("Input arguments: "+"\n host: "+host"+"\n port: "+port+"\n index: "+index+"\n path_to_ca_crt: "+path_to_ca_crt+"\n accessToken: "+accessToken);
String host=args[0];
String port=args[1];
String index=args[2];
String path_to_ca_crt=args[3];
String accessToken=args[4];
Path caCrtificatePath=Path.get(path_to_ca_crt);
CertificateFactory certFactory=
CertificateFactory.getInstance("X.509");
java.security.cert.Certificate trustedCa;
try (InputStream is=Files.newInputStream(caCrtificatePath)){
trustedCa=certFactory.generateCertificate(is);
}
KeyStore trustStore=keyStore.getInstance("pkcs12");
trustStore.load(null,null);
trustStore.setCertificateEntry("ca", trustedCa);
SSLContextBuilder sslContextBuilder=SSLContexts.custom()
.loadTrustMaterial(trustStore, null);
final javax.net.ssl.SSLContext sslContext=sslContextBuilder.build();
HostnameVerifier hostnameVerifier=NoopHostnameVerifier.INSTANCE;
SSLConnectionSocketFactory sslSocketFactory=
new SSLConnectionSocketFactory(sslContext, hostnameVerifier);
SchemeIOSessionStrategy httpsIOSessionStrategy=
new SSLIOSessionStrategy(sslContext, hostnameVerifier);
JestClientFactory factory=new JestClientFactory();
factory.setHttpClientConfig(new HttpClientConfig.Builder("https://"+host+":"+port)
.defaultSchemeForDiscoveredNodes("https")
.sslSocketFactory(sslSocketFactory)
.httpsIOSessionStrategy(httpsIOSessionStrategy)
.build()
);
JestClient client=factory.getObject();
String query="";
Search search=null;
Map<String,Object> header=new HashMap<>;
header.put("Authorization", "Basic "+accessToken);
search=new Search.Builder(query).addIndex.setHeader(header).build();
JestResult result=client.execute(search);
JsonObject jo=result.getJsonObject();
System.out.println("search result \n "+jo.toString());
}
}
Running project as Maven build and goal assempbly:single will produce JestToSecurES.jar with main class JestConnSecurES in its MANIFEST.MF file
Running/Testing java client on Linux
Notes:
-Place http.crt (mentioned above in this post) along with JestToSecurES.jar in particular directory
-Create/generate value of accessToken by encoding the value of auth_user:auth_user_password in Base64
-The index books has been initially created in the cluster and as previously noted the authenticated user has been granted permissions to monitor the cluster and all operations on indices
Allowed accessToken
java -jar JestToSecurES.jar [host_or_ip] [port] books http.crt [Base64(auth_user:auth_user_password)]
Console output will contain: ... "total":1,"successfull":1 ... "_index":"books","type":"book" ...
Forbidden accessToken (change previous value of [Base64(auth_user:auth_user_password)] somehow to corrupt it)
java -jar JestToSecurES.jar [host_or_ip] [port] books http.crt [Base64(auth_user:auth_user_password)]
Console output will contain: ... "error":{"root_cause"} ... "security_exception" ... "unable to authenticate user" ... for REST request [/books/_search]...
Goals:
-setup/configure fully secured elastic cluster with X-Pack (TLS/SSL for internal and external communication) on RHEL
-implement RBAC for authenticated user
-implement simple java client based on JEST API and test it with secured cluster
Summary of proceeded steps to achieve the goals
[1]Setup and configured 2 host (two nodes on each) ES-7.8.0 unsecured cluster on Linux
-Created index books
-Tested successfully unsecured cluster with curl
[2]Enabled x-pack on the cluster and configured TLS/SSL for internal communication between nodes (elasticsearch.yml)
-Implemented Role Based Access Control
--Created authenticated user, user password via elastic api commands (not using Kibana), created particular role
--Granted permissions to particular role and assigned the authenticated user for this role
-Tested successfully cluster with curl to make sure RBAC works as expected
[3]Configured TLS/SSL on HTTP to being able to make request with httpS
--changed elasticsearch.yml accordingly
--generated elastic-certificate.p12 with bin/elasticsearch-certutil
--extracted certificate from elastic-certificate.p12 into http.p12 with openssl
-Tested successfully fully secured cluster with curl
[4]Created simple java app based on JEST API
-Tested successfully fully secured elastic cluster with simple java client
To implement java client on JEST, reading elastic doc HLRC/LLRC allowed me to understand how to incorporate http.crt into java jest client.
I'll try to implement java client on HLRC/LLRC later on.
I agree with you, it is not easy to consult developers on this topic with different issues, because there are lots of dependencies like releases, used libs, system platforms, developers work on, different terminologies/meanings used for the same things or vice versa, etc., regardless elastic/x-pack doc is very detailed and contains lots of cross-references.
I think having simple use case sample for HLRC/LLRC java client with the steps I proceeded above from end-to-end, will allow developers to proceed with everything efficient and make the process more understandable and enjoyable.
Thank you all, elastic team for sharing your experience/knowledge globally and your help and support.
Next task is to make hadoop-elasticsearch/connector works with this TLS/SSL x-pack secured elastic cluster.