Logstash don't reach elasticsearch

Elastic Stack Logstash
1

Hi all,

I set up an elasticsearch's cluster with 3 nodes. I deploy the first one e installed after kibana in the same VM(RHEL).

It's work ok. With Xpack-security enable.

elasticsearch.yml:

image
image933×759 166 KB

Added other 2 nodes using enrolment token.

Everything is ok.

When I set up logstash and try connect to elasticsearch I saw this error:

image
image1865×116 104 KB

Telnet to elasticsearch on port 9200 work well.

Basic Security Logstash

Created role logstash_writer e logstash_read e logstash_internal's user that elastic.co recommend, howerver still don't work.

I believe one error in elasticsearch, because metricbeat didn't work too.

I didn't change any thing in logstash.yml.

Follow file .conf that I used:

image
image1859×839 177 KB

Regards,

2

can you execute

curl -u user:pass -XGET "http://<hostname>:9200/_cluster/health?pretty

use same username password you trying on logstash?
how about use user elastic?

3

image
image1465×422 223 KB

Show this error: curl:(52) Empty reply from server.

image
image1913×569 126 KB

Inside Kibana woring well.

I don't see any problem to use elastic. I am using logstash_internal because I followed elastic procedure.

4

Have you tried with https ? You have tls enabled, but you are communicating to elasticsearch using http.

5

image
image1466×794 154 KB

@Ugo_Sangiorgi still don't working.

6

I need create a self certificated ? How I do it ?

7

https://discuss.elastic.co/t/how-to-replace-security-certificate-sign-by-elastic-ca/302840

Check this out

8
  1. your ES uses a self signed certificate, so your curl must include "-k" for insecure
  2. check your installation, you will find a file called "http_ca.crt" under elasticsearch-/config/certs
9

@Ugo_Sangiorgi @elasticforme

I have created CA without password. After that set up my elasticsearch.yml with xpack security enable and redefine transport.ssl path.

However elasticsearch didn't start because this error:

image
image1895×196 141 KB

When I made yum install in the final show me elastic password:

image
image1211×196 52 KB

I tried the same procedure passing this password but didn't work too.

Without security cluster work well.

Is there something that I make wrong ?

10

When I create cluster I do execute this command before doing anything and setup my password.
then restart cluster with adding my ssl files in config etc..

/usr/share/elasticsearch/bin/elasticsearch-setup-passwords interactive

11

@elasticforme

I use these password when I will create CA and certificate ? Or still use empty password ?

Regards,

Silas Muniz

12

that password and certificate is two different thing.
certificate is used to communicate with other node in cluster

I know it is hard to get head around it if you doing first time security thing. as it happened to me.

you have already created cluster and has these lines in your elasticsearch.yml file then you are already using certificate. something like this

xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: /etc/elasticsearch/config/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: /etc/elasticsearch/config/elastic-certificates.p12

now you are using some use logstash_internal but I don't know what role you have for it.
first use user elastic and see it it work.
if does then it is user grant problem.

13

@elasticforme

First of all, thank you for you help.

I use elastic credential in logstash but not work. Kibana accpeted elastic credential for authentication.

What's I do wrong ?

I have seen this error in logstash:

< [INFO ] 2022-08-31 15:09:25.218 [Ruby-0-Thread-9: :1] elasticsearch - Failed to perform request {:message=>"192.168.0.35:9200 failed to respond", :exception=>Manticore::ClientProtocolException, :cause=>#<Java::OrgApacheHttp::NoHttpResponseException: 192.168.0.35:9200 failed to respond>}

[WARN ] 2022-08-31 15:09:25.219 [Ruby-0-Thread-9: :1] elasticsearch - Attempted to resurrect connection to dead ES instance, but got an error {:url=>"http://elastic:xxxxxx@192.168.0.35:9200/", :exception=>LogStash::Outputs::Elasticsearch::HttpClient::Pool::HostUnreachableError, :message=>"Elasticsearch Unreachable: [http://192.168.0.35:9200/][Manticore::ClientProtocolException] 192.168.0.35:9200 failed to respond"} />

I used /usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic to change elastic password.

Regards

14

@Ugo_Sangiorgi @elasticforme

curl -u elastic:kKq46rm5qwkjwkNiTd -XGET "https://localhost:9200/_cluster/health?pretty" -k
{
"cluster_name" : "my-application",
"status" : "green",
"timed_out" : false,
"number_of_nodes" : 1,
"number_of_data_nodes" : 1,
"active_primary_shards" : 16,
"active_shards" : 16,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 0,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0,
"task_max_waiting_in_queue_millis" : 0,
"active_shards_percent_as_number" : 100.0

15

your curl for cluster health works that means user/password is connecting to elasticsearch.

error before is showing something different. it has IP address 192.168.0.35 while you testing curl with localhost

16

@elasticforme

The error is the same, only change is that in the top a use a production enviroment(ip 172.20.6.206) and the last post I am using the lab enviroment(192.68.0.35).

Logstash => 192.168.0.39
Elasticsearch => 192.168.0.35

I try implement security in my stack because a need use observality for use alert of my logs.
I want create a rule to analyse my logs and send it for other server. For instance, when um rule reached a condition this log will send to other server(SIEM for example).

Regards

17

Hello everyone,

Silas you need to understand that if you implement ssl your instances will use the "https" instead of "http" this could mean a lot of things but first you have to switch your configuration to use "https"

Elasticsearch will not answer you on HTTP ! please use HTTPS

You also need to ensure connectivity between logstash and elastic that's why doing the curl request from localhost is useless unless you just try the syntax of the command.

Please perform a request from logstash to elastic with curl.

You can also use this bash tricks which check if the port is open at the TCP level :

echo > /dev/tcp/ELASTIC_IP/9200 && echo "OK"
18

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.