Confirm my steps on X-pack usage

Elastic Stack Elasticsearch
1

Hi ,

I have used x-pack in my ELK setup and i was able to achieve the secured communication. I will mention the steps that i have performed. Please tell me if i have followed it right.

step1:
i have installed x-pack plugin on elasticsearch using the below command,
elasticsearch-plugin.bat install x-pack

step2:
generated the certificates using the x-pack binaries certgen.bat

step3:
Add the certificate path in elasticsearch.yml ,kibana.yml and logstash.config as below

elasticsearch.yml
xpack.security.transport.ssl.enabled: true
xpack.security.http.ssl.enabled: true
xpack.security.audit.enabled: true

xpack.ssl.key: "E:/softwares/ELK/elasticsearch-5.4.0/elasticsearch-5.4.0/config/x-pack/newCert26June/ControlNode/ControlNode.key"

xpack.ssl.certificate: "E:/softwares/ELK/elasticsearch-5.4.0/elasticsearch-5.4.0/config/x-pack/newCert26June/ControlNode/ControlNode.crt"

xpack.ssl.certificate_authorities: "E:/softwares/ELK/elasticsearch-5.4.0/elasticsearch-5.4.0/config/x-pack/newCert26June/ca/ca.crt"

kibana.yml

elasticsearch.url: "https://192.168.0.104:9200"

elasticsearch.username: "elastic"
elasticsearch.password: "changeme"

server.ssl.enabled: true
server.ssl.certificate: "E:/softwares/ELK/elasticsearch-5.4.0/elasticsearch-5.4.0/config/x-pack/newCert26June/ControlNode/ControlNode.crt"
server.ssl.key: "E:/softwares/ELK/elasticsearch-5.4.0/elasticsearch-5.4.0/config/x-pack/newCert26June/ControlNode/ControlNode.key"

elasticsearch.ssl.certificateAuthorities: [ "E:/softwares/ELK/elasticsearch-5.4.0/elasticsearch-5.4.0/config/x-pack/newCert26June/ca/ca.crt" ]

logstash.config

output {
elasticsearch { hosts => ["192.168.0.104:9200"]
user => ["elastic"]
password => ["changeme"]
ssl => true
cacert => "E:/softwares/ELK/elasticsearch-5.4.0/elasticsearch-5.4.0/config/x-pack/newCert26June/ca/ca.crt"
retry_initial_interval => 5
retry_max_interval => 50
}
stdout { codec => rubydebug }

Upon performing the above steps, i was able to login to elasticsearch and kibana using https and also using cerdentials.

Please let me know if i have followed the steps properly and let me know i i am missing anything here.

2

My only suggestion would be to change the elastic password if you already haven't.

3

@joshbressers .. Thanks.. Will do that..

4

If you are installing X-Pack into Elasticsearch I think you should also install it into Kibana and Logstash - see steps 5 and 7 in the installation instructions. Maybe you already did this, as everything seems to be working, but you didn't explicitly list these steps in your question.

5

I have not installed x-pack in kibana and logstash. Only in elasticsearch i have installed. But still the flow is working fine. Can you please tell how will it impact if i dont install x-pack in kibana and logstash.

6

I can see from your original question that you have security enabled on Elasticsearch. Therefore, when you use Kibana you should have to enter a username and password configured in Elasticsearch before you can access any data. I'm guessing that at the moment the browser is popping up a very basic username/password dialog and you're logging in through that the first time Kibana tries to query data from Elasticsearch? This isn't the intended user experience. If you install X-Pack into Kibana then the login will be done using a login page that's part of the Kibana X-Pack plugin. There may also be more subtle problems - we don't test or recommend running with X-Pack inconsistently installed so can't easily say what these would be.

The other benefit to installing X-Pack into Kibana is that you'll get to use the UI pages for the X-Pack features, such as monitoring, user/role management, and (depending on your license level), watcher, graph and machine learning.

7

@droberts195.. Thansk for the info.. ok now. i Have installed x-pack on kibana, It shows a login page. i am logging using "elastic/changeme". But when i install x-pack on logstash, i am getting the following error,

logstash.outputs.elasticsearch - Attempted to resurrect connection to dead ES instance, but got an error. {:url=>#<URI::HTTP:0x504bedf2 URL:http://logstash_system:xxxxxx@localhost:9200/_xpack/monitoring/?system
_id=logstash&system_api_version=2&interval=1s>, :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [http://logstash_system:xxxxxx@localhost:
9200/][Manticore::SocketException] Connection refused: connect"}

My question is why it is trying to login using http://logstash_system:xxxxxx@localhost:9200.. Actually in logstash.config, i have given the elasticsearch username is "elastic/changeme" as below,

output {
elasticsearch {
hosts => ["132.186.102.153:9200"]
user => "elastic"
password => "changeme"
ssl => true
cacert => "D:/Softwares/ELK/elasticsearch-5.4.0/elasticsearch-5.4.0/config/x-pack/NewCert27June/ca/ca.crt"

}
stdout { codec => rubydebug }

}

Please help me here

8

The output section of logstash.config is where you declare the elasticsearch configuration for sending data on your data pipeline.

If x-pack is installed, then logstash will also send monitoring data to an elasticsearch cluster, which must also be configured. It's not possible to just use the same configuration as the one in your logstash.config since you can use logstash without sending to ES at all, or even have multiple elasticsearch output blocks.

To configure the ES cluster for the x-pack on logstash you can check the documentation in this link. Here's an example of what to change in logstash.yml:

xpack.monitoring.elasticsearch.url: ["http://es-prod-node-1:9200", "http://es-prod-node-2:9200"] 
xpack.monitoring.elasticsearch.username: "logstash_system" 
xpack.monitoring.elasticsearch.password: "changeme"
xpack.monitoring.elasticsearch.ssl.ca: [ "/path/to/ca.crt" ]
9

Ask for advice on this information.

10

@jsvd,

Thanks a lot for the info. I modified logstash.yml with the example you provided and it works now.
I have 2 more doubts.

  1. What exactly is the logstash monitoring data? Is it important to capture monitoring data and send to elasticsearch.

  2. I am interesting in only sending log file data to elasticsearch. So in this case, can i ignore installing x-pack in logstash.
    Why i am asking this because, I have a setup where a central machine is connected to so many machines in a distributed environment. All the distributed machines sends logs information to the central machine.
    Kibana and elasticsearch is running on central machine. logstash and filebeat is running on individual distributed machine. Each logstash instance connects to central machine's elasticsearch.
    Since x-pack is not free, if i end up installing x-pack per logstash, then the cost will rise. I have more than 100 machines in distributed environment. I am not aware of x-pack licence cost and also now aware on how the licence is issued. If its per machine basis, then installed x-pack for logstash per machine becomes a costly affair. Since elasticsearch and kibana lies in the same machine, i can considering installing x-pack for kibana and elasticsearch.

11

x-pack in logstash will capture information about event throughput, hot_threads, per plugin stats, persistent queue capacity stats, etc. This information can then be visible in kibana, in the monitoring console, alongside stats about elasticsearch and kibana.

Monitoring for logstash through x-pack is in the basic license, so it's free.

That said, you can always uninstall x-pack with bin/logstash-plugin uninstall x-pack

12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.