Elasticsearch.Net NEST connection string when Active Directory turn on (Elasticsearch Authentication)

Elastic Stack Elasticsearch
1

We have a cluster running Elasticsearch v5.0.0. Recently active directory functionality is turn on in ES so, the way connection string was written earlier it is not working and giving following error when perform Search/ Insert operation from .NET client. Could anyone give suggestion in what way the connection string should be written to authenticate with AD.

Code:

using Elasticsearch.Net;
using Nest;

ConnectionSettings connectionSettings;
ElasticClient elasticClient;
StaticConnectionPool connectionPool;

var nodes = new Uri[]
{
new Uri("http://username:password@1.2.3.4:9200/"),
new Uri("http://username:password@1.2.3.5:9200/"),
new Uri("http://username:password@1.2.3.6:9200/")
};

connectionPool = new StaticConnectionPool(nodes);
connectionSettings = new ConnectionSettings(connectionPool);
elasticClient = new ElasticClient(connectionSettings);

Error:

Invalid NEST response built from a unsuccessful low level call on POST: /twitter/boo/_search

Audit trail of this API call:

  • [1] PingFailure: Node: http://:@1.2.3.4:9200/ Exception: PipelineException Took: 00:00:00.0390039

OriginalException: Elasticsearch.Net.ElasticsearchClientException: One or more errors occurred. ---> System.AggregateException: One or more errors occurred. ---> Elasticsearch.Net.PipelineException: Failed to ping the specified node. ---> Elasticsearch.Net.PipelineException: An error occurred trying to read the response from the specified node.

at Elasticsearch.Net.RequestPipeline.Ping(Node node) in C:\code\elasticsearch-net\src\Elasticsearch.Net\Transport\Pipeline\RequestPipeline.cs:line 247
--- End of inner exception stack trace ---
at Elasticsearch.Net.RequestPipeline.Ping(Node node) in C:\code\elasticsearch-net\src\Elasticsearch.Net\Transport\Pipeline\RequestPipeline.cs:line 254
at Elasticsearch.Net.Transport1.Ping(IRequestPipeline pipeline, Node node) in C:\code\elasticsearch-net\src\Elasticsearch.Net\Transport\Transport.cs:line 179 at Elasticsearch.Net.Transport1.Request[TReturn](HttpMethod method, String path, PostData`1 data, IRequestParameters requestParameters) in C:\code\elasticsearch-net\src\Elasticsearch.Net\Transport\Transport.cs:line 67
--- End of inner exception stack trace ---
--- End of inner exception stack trace ---

Audit exception in step 1 PingFailure:

Elasticsearch.Net.PipelineException: An error occurred trying to read the response from the specified node.
at Elasticsearch.Net.RequestPipeline.Ping(Node node) in C:\code\elasticsearch-net\src\Elasticsearch.Net\Transport\Pipeline\RequestPipeline.cs:line 247

Request:

<Request stream not captured or already read to completion by serializer. Set DisableDirectStreaming() on ConnectionSettings to force it to be set on the response.>

Response:

<Response stream not captured or already read to completion by serializer. Set DisableDirectStreaming() on ConnectionSettings to force it to be set on the response.>

2

It looks like you're just using Basic Authentication and sending the username/password as part of the URI. Have you tried

using Elasticsearch.Net;
using Nest;

var nodes = new [] {  
    new Uri("http://1.2.3.4:9200/")  
};

var connectionPool = new StaticConnectionPool(nodes);
var connectionSettings = new ConnectionSettings(connectionPool)
    .BasicAuthentication("username", "password");

var elasticClient = new ElasticClient(connectionSettings);

Since you only have a single node, you can also use SingleNodeConnectionPool

var connectionPool = new SingleNodeConnectionPool(new Uri("http://1.2.3.4:9200/"));
3

Actually we have multiple nodes. In that case "SingleNodeConnectionPool" will work? or any other?

4

SingleNodeConnectionPool is only for clusters made up of a single node, so typically a non-production/development environment.

For multiple nodes, use StaticConnectionPool or SniffingConnectionPool. Take a look at the documentation on connection pooling.

5

"SingleNodeConnectionPool" solves the issue. Because our QA has one node for now.

6

Can you tell me why "StaticConnectionPool" cannot working after activating Active Directory? Because earlier when AD was not active single node with "StaticConnectionPool" was working.

7

Did you try using .BasicAuthentication() on ConnectionSettings?

By enabling Active Directory, have firewall settings on the machine been changed?

To rule out it being any issue with NEST, can you ping the cluster from the machine that is using NEST to connect to Elasticsearch, to see if the cluster can be reached. Use curl or PowerShell's Invoke-RestMethod for this.

8

Yes. Username and Password is passed there. And ping is working cluster can be rechable.

But this is not working with multiple node connection pooling. Using SingleNodeConnectionPoll can connect one node. But I need connection with fail over. That is not working with the above code in the first that I posted.

9

Did you try this?

Have you modified this audit trail message to remove the username/password? If not, it looks like your username/password are empty strings.

You can also enable network tracing for your application if you need more information on why the ping fails.

Are all the Uri passed to StaticConnectionPool correct? Can you ping each from the application box (to rule out an issue with the network)?

10

Yes. Just replace user and pass with dummy name. but somehow it is not showing. Message like this

Audit trail of this API call:

11

One solution I found if I disable the Ping. The search query to get data is working. Like this:

ConnectionSettings connectionSettings;
ElasticClient elasticClient;
StaticConnectionPool connectionPool;

var nodes = new Uri[]
{
new Uri("http://10.71.1.124:9200/"),
new Uri("http://10.71.1.125:9200/")
};

connectionPool = new StaticConnectionPool(nodes);
connectionSettings = new ConnectionSettings(connectionPool).BasicAuthentication("johndoe", "123456").DisablePing();
elasticClient = new ElasticClient(connectionSettings);

return elasticClient;

In the above code if I disable Ping, it is working. Can this be a solution?
Any idea why ping causing issue?

12

You can disable ping for a StaticConnectionPool, but that still doesn't explain why pinging a node in the cluster is failing.

  • Have you enabled network tracing and looked at the log?
  • Are HEAD requests to http://1.2.3.4:9200/ allowed?
  • what version of NEST are you using?
13

Also looked into deeper "SingleNodeConnectionPool" has by default pinging disable where "StaticConnectionPool" has enable.

https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/pinging-first-usage.html

No, haven't enable network trace yet.

14

You should upgrade NEST and Elasticsearch.Net to a 5.x version (latest at this time is 5.0.1); Major versions of the client are only compatible with major versions of Elasticsearch. Whilst they may work for some things, there are known differences that will not be compatible.

Would you be able to upgrade these and check if you still have the ping issue with StaticConnectionPool?

15

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.