Java client 8.2.2 Trouble loading keystore during setup

Elastic Stack Elasticsearch
1

I created a gist here

When doing a keystore.load I get
DerInputStream.getLength(): lengthTag=87, too big.

Apparently that's a rich topic, as in

Most of the issue seems to be on platforms other than ES.
I'm hoping for some insight about what's going on.
I am pointing to the cert provided in the /config/certs directory.

2

@Jack_Park Did you ever get this solved?

3

Nope.
Thus far @stephenb , you're the only response.
I upgraded it to an issue at the repo,.
Thus far, nobody has responded there either.

You know, if there were a folder in the repo called /examples with fully expressive examples of several ways to boot the client. that would be great, Instead, for now, there may be a dozen or more different places around the web one has to go, and not a single one of them covers all bases.

In a different but partially related issue, I learned that there is an error in the file which is used to generate client code and it will be fixed in the next release, but that is not related to the "too long" tag when loading the keystore. Nobody here appears to want to take it on, I get that'; I may be the only person reporting it.

Thanks

4

I reached back into the org .. see if I can get some help... Surprised it's so difficult.

@dadoonet Are you java client literate?

5

@swallez In case you could help.

6

Hiya @Jack_Park

Initial internal quick look

Perhaps it's the issue with the trustore format. They are trying to load it in pkcs12 , but it's probably was generated in jks which is the default if you use Java's keytool

Have you confirmed that?

7

There's a file format error in the file at trustStorePath. Is it a .p12 file, which is expected by KeyStore.getInstance("pkcs12")? If this is a .jks file, the KeyStore should be created as "JKS".

That being said, you should not need to load anything in the keystore except the certificate. Here's a helper function that creates a working SSL context (tested with Elasticseach 8.2.2):

    public static SSLContext fromHttpCaCrt(File file) throws CertificateException, IOException {
        try(InputStream in = new FileInputStream(file)) {
            return fromHttpCaCrt(in);
        }
    }

    public static SSLContext fromHttpCaCrt(InputStream in) throws CertificateException {
        try {
            CertificateFactory pkcs1 = CertificateFactory.getInstance("X.509");
            Certificate certificate = pkcs1.generateCertificate(in);

            final KeyStore keyStore = KeyStore.getInstance("pkcs12");
            keyStore.load(null, null);
            keyStore.setCertificateEntry("elasticsearch-ca", certificate);

            TrustManagerFactory tmf = TrustManagerFactory.getInstance("X.509");
            tmf.init(keyStore);

            SSLContext sslContext = SSLContext.getInstance("TLS");
            sslContext.init(null, tmf.getTrustManagers(), null);

            return sslContext;
        } catch (NoSuchAlgorithmException | KeyManagementException | KeyStoreException | IOException e) {
            throw new RuntimeException(e);
        }
    }
1 Like
8

Thanks very much.
You're now plowing grounds on which I experimented. Cannot recall if I tried jks or not; I did try something other than pks12, to no avail.
Having said that, I did not personally construct the keystore; it was there after install.

The precise code I used to boot the client is that which I copied from an ES online document.

I shall now go in and try the code you just supplied.
Thanks!

9

The change to how keystore is loaded allowed it to go further into my test. The new error is

ERROR 2022-06-24 12:56:22,621 [main] - [es/indices.create] failed: [security_exception] missing authentication credentials for REST request [/topics]

We are beyond booting - that is, there is some reason to believe that the ES client booted without errors; we are now in the process of creating an index in this code

		IndexSettings.Builder isb = new IndexSettings.Builder();
		isb.numberOfShards(numberOfShards);
		isb.numberOfReplicas(numberOfReplicas);
		
		//@see https://github.com/elastic/elasticsearch-java/blob/66da097630dccc29da677f4a32ed8b468bceff3d/java-client/src/test/java/co/elastic/clients/json/WithJsonTest.java
		CreateIndexRequest.Builder b = new CreateIndexRequest.Builder()
				.index(indexName)
				.settings(isb.build())
				.withJson(new StringReader(mappings));
		try {
			CreateIndexResponse createIndexResponse = client.indices().create(b.build());
			result.setResultObject(new Boolean(createIndexResponse.acknowledged()));
		} catch (Exception e) {

where the error occurs at createIndexResponse

I cannot say precisely where I got that code but I have reason to believe I found pieces of it in ES test cases.

10

Ok, that problem was solved.

Let me reiterate my passionate request that ElasticSearch, once and for all, make some examples which cover all bases in one single place online. There are dozens of places to find stuff, a none of them coordinated. We already know about setting up ssl. Go elsewhere, e.g.

for setting up authentication, and ssl is missing. TRUTH IS: for 8.2.2, you have to do both, at the same time.

No mortal is going to wade through that mess.
Fact is, half the "8.2" pages have links to pages which don't exist anymore. So, one is forced to make google stay on the topic of 8.2.2 - nothing less will suffice.

At least now, it says here in the fine print, I have a working client -- with still more work to do to add all its functionality.

Many thanks Sylvain, for all your help.

11

@Jack_Park thank you for the feedback. And I'm glad you got this working

The error above now about the indices is an authorization error looks like which is about users and roles.

What would be most helpful if now you can post the snippet and steps clearly that you did so the next person can follow this.

Will try to get some of this put into the docs but you putting it here would be great help getting that started and for anyone else that comes after you, That's what we hope for in the community.

So that we can show the solution and mark it solved. So when someone's looking they'll be able to find it.

12

Will work on getting a snippet here. Will take a while.

1 Like
13

Sadly, as I continue working on the code, it's gone back to failing to authenticate on createIndex. I'll explain later with a gist.

14

@Jack_Park

Here is my code (I am no longer a Java Dev) so excuse the poor code.

I assembled this from the Basic Auth and Encrypted Communications pages ... I agree would be better if was a whole sample.

This using 8.2.3 with the default setup security set up so this is the self signed cert that was created when I first ran ./bin/elasticsearch
This is on my MacOS.

...
hyperion:es-client sbrown$ java -version
java version "11.0.13" 2021-10-19 LTS
Java(TM) SE Runtime Environment 18.9 (build 11.0.13+10-LTS-370)
Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11.0.13+10-LTS-370, mixed mode)
...
hyperion:certs sbrown$ pwd
/Users/sbrown/workspace/elastic-install/8.2.3/elasticsearch-8.2.3/config/certs
hyperion:certs sbrown$ ls -l
total 24
-rw-rw----  1 sbrown  staff  10189 Jun 24 19:39 http.p12
-rw-rw----  1 sbrown  staff   1915 Jun 24 19:39 http_ca.crt
-rw-rw----  1 sbrown  staff   5822 Jun 24 19:39 transport.p12

Perhaps this will help
It connect and lists the indices .. just 1 with the new cluster.

hyperion:es-client sbrown$  cd /Users/sbrown/workspace/demos/es-java-client/es-client ; /usr/bin/env /Library/Java/JavaVirtualMachines/jdk-11.0.13.jdk/Contents/Home/bin/java @/var/folders/p9/r3gjl2912ybd827msztppr_m0000gn/T/cp_5q9x8i9i9ha4wvw66s4gp3rsl.argfile com.example.App 
pom.xml         src/            target/
Hello World!
[{"health":"yellow","status":"open","index":"asset-index","uuid":"IiBjcXwZTIKQloHhinIUmg","pri":"1","rep":"1","docs.count":"2","docs.deleted":"0","store.size":"4.1kb","pri.store.size":"4.1kb"}]
hyperion:es-client sbrown$
15

So, my code looks precisely like the code in your gist.
Mind you, I am running on 8.2.2 - not even sure if 8.2.3 is available to me yet.

But, I get this exception

co.elastic.clients.elasticsearch._types.ElasticsearchException: [es/cat.indices] failed: [security_exception] unable to authenticate user [elastic] for REST request [/_cat/indices?format=json]

when I am executing a line from your code which I added:

String cat = client.cat().indices().toString();

Even there, authentication fails.

I am using the username "elastic" and the password given to me when I installed the system.

16

8.2.3 is now downloading to my MacBook. I'll try again with it.

17

I can now report that, with 8.2.3, the code is working just fine at least to the level of creating an index. New issues with mapping, but that's another story.

Many thanks!

1 Like
18

The error you are getting is failed authentication which means bad username and / or password.
It also means the SSL is working because you would have not got that far.

With the exact same creds have you tried the following.

curl --insecure -u "elastic:password" https://yourhost:port/_cat/indices

19

@Jack_Park

Sorry some reason my site didn't update. Glad you got it running.

I would make sure you know how to run every command from the Kibana Dev Tools first and make sure they work successfully before you try them through one of the language clients. Just my suggestion.

Glad you got past this

You may make me go out and try this on 8.2.2 see what's going on :slight_smile:

20

The curl returned

green open topics XcGafF3FQ46-CDD5xXM-0w 1 0 0 0 225b 225b

which is correct.

Now I'm off in another thread in this venue dealing with a parse error - an unrecognized field in my json data. Most likely I'm reliving an ancient battle I fought with the ES 6 clan.