Hi,
I installed Elastic and Kibana localy on my device and I create a index.
Than I create a C# console project using .NET6.0 and Visual Studio as described here.
But I can't get it running. The problem I described here.
Could it be that I need a Authentication to access Elastic when hosted locally?
Thanks,
Frank
Hello,
The problem page isn't available. It returns an error.
And, no, you don't need authentication in Elasticsearch if it's not configured.
For example, running some instances in docker:
version: '3.3'
services:
Elasticsearch:
image: elasticsearch:7.4.0
container_name: elasticsearch
environment:
- discovery.type=single-node
ports:
- 9200:9200
restart: unless-stopped
networks:
- netElk
Kibana:
image: kibana:7.4.0
container_name: kibana
ports:
- 5601:5601
restart: unless-stopped
networks:
- netElk
depends_on:
- Elasticsearch
Logstash:
image: logstash:7.4.0
container_name: logstash
ports:
- 5044:5044
restart: unless-stopped
networks:
- netElk
networks:
netElk:
driver: bridge
$ docker-compose up -d
Creating network "netElk" with driver "bridge"
Creating logstash ... done
Creating elasticsearch ... done
Creating kibana ... done
[2022-05-13 12:25:18] {afuscoar@afuscoar} (~/Documents)$ -> curl --insecure -X GET http://$(dockerip elasticsearch):9200/_cat/health
1652437535 10:25:35 docker-cluster green 1 1 0 0 0 0 0 0 - 100.0%
Make a conection using Python:
$ cat test_connection.py
from elasticsearch import Elasticsearch
es = Elasticsearch(['http://172.23.0.2:9200/'])
if not es.ping():
raise ValueError("Connection failed")
print(es.cluster.health())
$ python3 test_connection.py
{'cluster_name': 'docker-cluster', 'status': 'green', 'timed_out': False, 'number_of_nodes': 1, 'number_of_data_nodes': 1, 'active_primary_shards': 3, 'active_shards': 3, 'relocating_shards': 0, 'initializing_shards': 0, 'unassigned_shards': 0, 'delayed_unassigned_shards': 0, 'number_of_pending_tasks': 0, 'number_of_in_flight_fetch': 0, 'task_max_waiting_in_queue_millis': 0, 'active_shards_percent_as_number': 100.0}
Thank you Adrian,
but parallel I got Kibana open and there I can access Elastic using the console.
(URL: http://localhost:5601/app/dev_tools#/console)
So I don't understand why the page isn't available?!?
PS: I don't use Docker.
Mmm, strange.
Depending of the version it could be another URL, e.g in my case it's: http://localhost:5601/app/kibana#/dev_tools/console?_g=()
Do you have access in http://localhost:5601 ? Try to access using the left bar to the dev tools.
Yes. I can call http://localhost:5601 and reach Kibana.
And in Kibana I can reach Elastic, write and read on it.
Accessing the Kibana URL works fine.
var client = new WebClient();
var content = client.DownloadString("http://localhost:5601");
@frankmehlhop What error are you seeing? By default, the client will connect to http://localhost:9200 where Elasticsearch should be listening by default. Which version of Elasticsearch have you installed? v8.0+ start with HTTPS configured and authentication enabled.
@stevejgordon The Error message I get is:
Invalid NEST response built from a unsuccessful () low level call on POST: /test/_doc Audit trail of this API call:
1.Ping(IRequestPipeline pipeline, Node node) at Elasticsearch.Net.Transport 1.Request[TResponse](HttpMethod method, String path, PostData data, IRequestParameters requestParameters) --- End of inner exception stack trace --- Request: <Request stream not captured or already read to completion by serializer. Set DisableDirectStreaming() on ConnectionSettings to force it to be set on the response.> Response: <Response stream not captured or already read to completion by serializer. Set DisableDirectStreaming() on ConnectionSettings to force it to be set on the response.>Hi Steve,
I use version 8.x
Where can I set HTTPS configured and authentication enabled?
I can't figure that out. At Kibana, on the shell or at my C# code?
You got a link where I find a description for it?
Would be so helpful. Thank you.
Good morning @frankmehlhop.
Using the 8.x requires some extra configuration to enable HTTPS, authentication and a compatibility mode when using the latest 7.17 client. This is because the v8 client starts secure by default so the client must trust its certificate and authenticate with a valid user or API key. This guide from our documentation should get you up and running.
I've just noticed an error in that documentation. The Uri should be "https://localhost:9200". I'll fix that now!
Hi @stevejgordon,
thanks a lot for your message.
I tried using the code a you recommended.
I set the environment variable on my device ELASTIC_CLIENT_APIVERSIONING to "1" or "true". (tried both)
But I still get the same error at the indexresponce -object on the end of method InjectData().
I find also a documentation from elastic about Configuration File, but the mentioned file curator.yml does not exist in my installation. And I searched all files at my ELK-Stack folder for 'ELASTIC_CLIENT_APIVERSIONING', but I got no hits.
The password I used in BasicAuthentication() works fine at Kibana.
The index "test" exists and has already a entry (done using Kibana DevTools / console).
I also tried using the LowLevelClient and the responseByte ist empty (code you find below).
Where is the mistake / problem on my code or settings?
(German language settings)
using Elasticsearch.Net;
using Nest;
namespace ElasticTest
{
public class Manager
{
readonly ElasticClient client;
public Manager()
{
var pool = new SingleNodeConnectionPool(new Uri("https://localhost:9200"));
var settings = new ConnectionSettings(pool)
.CertificateFingerprint("94:75:CE:4F:EB:05:32:83:40:B8:18:BB:79:01:7B:E0:F0:B6:C3:01:57:DB:4D:F5:D8:B8:A6:BA:BD:6D:C5:C4")
.BasicAuthentication("elastic", "myPassword")
.EnableApiVersioningHeader()
.DefaultIndex("test");
client = new ElasticClient(settings);
}
public bool InjectData()
{
Person person = new Person
{
name = "Anna",
age = 29,
address = "Beethovenweg 8, 99817 Eisenach"
};
IndexResponse indexResponse = client.IndexDocument(person);
return indexResponse != null;
}
}
}
public bool LowLevelClient()
{
var settings = new ConnectionConfiguration(new Uri("https://localhost:9200"))
.RequestTimeout(TimeSpan.FromSeconds(5));
var lowlevelClient = new ElasticLowLevelClient(settings);
var indexResponse = lowlevelClient.Index<BytesResponse>("test", "123", PostData.Serializable(GetPerson()));
byte[] responseBytes = indexResponse.Body;
return responseBytes.Length > 0;
}
@frankmehlhop The code for the high-level client at first glance looks valid. Can you reach "https://localhost:9200" on your device? Do you get prompted for credentials? If so, do you credentials get accepted allowing an info response from the server to be returned. If so, can you please provide the output?
How did you install Elasticsearch?
Hi @stevejgordon,
I can answer your questions with yes.
the response:
{
"name" : "DESKTOP-BA7xxxx",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "t8ZCkbloTvK_Mkh7-PELwg",
"version" : {
"number" : "8.1.3",
"build_flavor" : "default",
"build_type" : "zip",
"build_hash" : "39afaa3c0fe7db4869a161985e240bd7182d7a07",
"build_date" : "2022-04-19T08:13:25.444693396Z",
"build_snapshot" : false,
"lucene_version" : "9.0.0",
"minimum_wire_compatibility_version" : "7.17.0",
"minimum_index_compatibility_version" : "7.0.0"
},
"tagline" : "You Know, for Search"
}
For installing I started on my Windows 11 cmd this file:
C:\ELK-Stack\elasticsearch-8.1.2-windows-x86_64\elasticsearch-8.1.2\bin\elasticsearch.bat
What else could I check?
I also tried to use the lowLevelClient and I tried a simple Ping and I always get the same response.
PingResponse pingResponse = client.Ping();
response:
OriginalException: Elasticsearch.Net.ElasticsearchClientException: The client is unable to verify that the server is Elasticsearch due to an unsuccessful product check call. Some functionality may not be compatible if the server is running an unsupported product. Call: Status code unknown from: GET /
---> Elasticsearch.Net.PipelineException: The client is unable to verify that the server is Elasticsearch due to an unsuccessful product check call. Some functionality may not be compatible if the server is running an unsupported product.
---> System.Net.Http.HttpRequestException: An error occurred while sending the request.
---> System.IO.IOException: The response ended prematurely.
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
That's rather strange. The exception seems to be down in the .NET HTTP code, "System.IO.IOException: The response ended prematurely". This was also appearing in your previous stack trace, but in that case likely due to using the http URL. Can you share the full DebugInformation on the response once again? You may also be able to run a tool such as Fiddler to capture the localhost requests and responses which may help understand what's going on.
Hi @stevejgordon,
thanks for checking this case.
I got nearly no experience with fiddler.
Is there a tutorial using fiddler for elastic?
Thanks!
here the full message from Ping response:
(and it doesn't matter I use http or https.)
Invalid NEST response built from a unsuccessful () low level call on HEAD: /
Audit trail of this API call:
- [1] ProductCheckOnStartup: Took: 00:00:01.7516949
- [2] ProductCheckFailure: Node: http://localhost:9200/ Took: 00:00:01.1677262
OriginalException: Elasticsearch.Net.ElasticsearchClientException: The client is unable to verify that the server is Elasticsearch due to an unsuccessful product check call. Some functionality may not be compatible if the server is running an unsupported product. Call: Status code unknown from: GET /
---> Elasticsearch.Net.PipelineException: The client is unable to verify that the server is Elasticsearch due to an unsuccessful product check call. Some functionality may not be compatible if the server is running an unsupported product.
---> System.Net.Http.HttpRequestException: An error occurred while sending the request.
---> System.IO.IOException: The response ended prematurely.
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
at Elasticsearch.Net.HttpConnection.Request[TResponse](RequestData requestData)
--- End of inner exception stack trace ---
at Elasticsearch.Net.RequestPipeline.ThrowIfTransientProductCheckFailure()
at Elasticsearch.Net.RequestPipeline.Ping(Node node)
at Elasticsearch.Net.Transport1.Ping(IRequestPipeline pipeline, Node node) at Elasticsearch.Net.Transport1.Request[TResponse](HttpMethod method, String path, PostData data, IRequestParameters requestParameters)
--- End of inner exception stack trace ---Request:
<Request stream not captured or already read to completion by serializer. Set DisableDirectStreaming() on ConnectionSettings to force it to be set on the response.>
Response:
<Response stream not captured or already read to completion by serializer. Set DisableDirectStreaming() on ConnectionSettings to force it to be set on the response.>
@frankmehlhop. That error suggests that the client is still configured to use "http://localhost:9200" instead of "https://localhost:9200" which is required in this case. Can you check your code to ensure the settings are being passed into the client. I've attempted to reproduce with a ZIP install and it connects without error, so this looks like a configuration issue.
Also, have you replaced the "CertificateFingerprint" string with the one matching your server? That is an example value which will be different for every installation. Also, make sure you've replaced the password as well.
Based on your error though, this very much looks like it's attempting to connect over HTTP. not HTTPS and the server is closing the connection. This is also indicated by the node used for step two of the audit trail. Again, this looks like your client instance hasn't been created using the correct settings instance. It's hard to provide any further guidance as this appears to be a local issue.
Fiddler is a general tool. The client will proxy requests through it when it is running. There are no Elasticsearch specific guides available. With HTTPS it's actually a little involved to run it, so let's not worry about this here. I'd recommended debugging your code to ensure that the settings has the expected value when the client instance is created. You can navigate to the non-public properties of the client to eventually access the settings instance and check the values there.
Hi @stevejgordon,
I have reinstalled elastic + kibana and now access works.
Thank you very much for your time and best regards!
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
© 2020. All Rights Reserved - Elasticsearch
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.