X-Pack et logstash

Elastic In Your Native Tongue Discussions en français
1

Bonjour,

je viens d'installer X-PACK sur un POC Elasticsuite.

Je n'arrive plus à faire communiquer logstash avec ES.

J'ai pourtant mis à jour les mots de passe pour les comptes logstash_system, ainsi que logstash (role logstash_writer)

Voici l'erreur que j'obtiens à la relance de logstash :

[2017-04-18T17:11:46,504][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://logstash_system:xxxxxx@127.0.0.1:9200/, :path=>"/"}
[2017-04-18T17:11:46,648][WARN ][logstash.outputs.elasticsearch] **Attempted to resurrect connection to dead ES instance, but got an error.** {:url=>#<URI::HTTP:0x11429f09 URL:http://logstash_system:xxxxxx@127.0.0.1:9200/>, :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError, :error=>"Got response code '401' contacting Elasticsearch at URL 'http://127.0.0.1:9200/'"}
[2017-04-18T17:11:46,649][INFO ][logstash.outputs.elasticsearch] Using mapping template from {:path=>nil}
[2017-04-18T17:11:46,804][ERROR][logstash.outputs.elasticsearch] Failed to install template. {:message=>"Got response code '401' contacting Elasticsearch at URL 'http://127.0.0.1:9200/'", :class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError", :backtrace=>["/produit/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-6.2.6-java/lib/logstash/outputs/elasticsearch/http_client/manticore_adapter.rb:76:in `perform_request'", "/produit/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-6.2.6-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:273:in `perform_request_to_url'", "/produit/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-6.2.6-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:261:in `perform_request'", "/produit/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-6.2.6-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:351:in `with_connection'", "/produit/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-6.2.6-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:260:in `perform_request'", "/produit/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-6.2.6-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:268:in `get'", "/produit/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-6.2.6-java/lib/logstash/outputs/elasticsearch/http_client.rb:83:in `get_version'", "/produit/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-6.2.6-java/lib/logstash/outputs/elasticsearch/template_manager.rb:16:in `get_es_version'", "/produit/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-6.2.6-java/lib/logstash/outputs/elasticsearch/template_manager.rb:20:in `get_es_major_version'", "/produit/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-6.2.6-java/lib/logstash/outputs/elasticsearch/template_manager.rb:7:in `install_template'", "/produit/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-6.2.6-java/lib/logstash/outputs/elasticsearch/common.rb:54:in `install_template'", "/produit/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-6.2.6-java/lib/logstash/outputs/elasticsearch/common.rb:21:in `register'", "/produit/logstash/logstash-core/lib/logstash/output_delegator_strategies/shared.rb:8:in `register'", "/produit/logstash/logstash-core/lib/logstash/output_delegator.rb:37:in `register'", "/produit/logstash/logstash-core/lib/logstash/pipeline.rb:282:in `register_plugin'", "/produit/logstash/logstash-core/lib/logstash/pipeline.rb:293:in `register_plugins'", "org/jruby/RubyArray.java:1613:in `each'", "/produit/logstash/logstash-core/lib/logstash/pipeline.rb:293:in `register_plugins'", "/produit/logstash/logstash-core/lib/logstash/pipeline.rb:302:in `start_workers'", "/produit/logstash/logstash-core/lib/logstash/pipeline.rb:232:in `run'", "/produit/logstash/logstash-core/lib/logstash/agent.rb:387:in `start_pipeline'"]}

Puis

[2017-04-18T16:39:57,627][ERROR][logstash.outputs.elasticsearch] Got a bad response code from server, but this code is not considered retryable. Request will be dropped {:code=>401, :response_body=>"{"error":{"root_cause":[{"type":"security_exception","reason":"failed to authenticate user [logstash_system]","header":{"WWW-Authenticate":"Basic realm="security" charset="UTF-8""}}],"type":"security_exception","reason":"failed to authenticate user [logstash_system]","header":{"WWW-Authenticate":"Basic realm="security" charset="UTF-8""}},"status":401}"}

J'ai testé une connexion directe : OK

curl --user logstash_system -XGET 'localhost:9200/_cluster/health'
Enter host password for user 'logstash_system':
{"cluster_name":"elasticsearch_prodj","status":"yellow","timed_out":false,"number_of_nodes":1,"number_of_data_nodes":1,"active_primary_shards":31,"active_shards":31,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":30,"delayed_unassigned_shards":0,"number_of_pending_tasks":0,"number_of_in_flight_fetch":0,"task_max_waiting_in_queue_millis":0,"active_shards_percent_as_number":50.8

La fin de mon fichier logstash.yml :

#Fichier logstash.yml
# Security X-PACK
xpack.monitoring.elasticsearch.url: http://localhost:9200
xpack.monitoring.enabled: true # False ne marche pas mieux
xpack.monitoring.elasticsearch.username: logstash_system
xpack.monitoring.elasticsearch.password: LE MOT DE PASSE

Les utilisateurs ont été créé via la console.

Est-il nécessaire de les créer en local via bin/x-pack/users ?

bref, je sèche un peu. Merci d'avance.

David

2

Tu as configuré logstash pour qu'il passe le user/password? https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-user

3

Bonjour,

Oui c'est configuré au niveau de chacun des output :

 elasticsearch {
      hosts => [ "127.0.0.1:9200" ]
      index => "rsyslog-%{+YY.MM}"
      user => "logstash_system"
      password => "XXXXXXXXXXXXX"
    }

Par contre je pense que ça se passe avant, au niveau du Healthcheck de logstash ?

Faut il créer un index .monitoring et ajouter les droits idoines pour le role logstash_system ? (ça devrait être built-in non ?)

David

EDIT : je me relis et je me rends compte que j'ai fait une connerie. (logstash_system n'est pas le compte censé ajouter des outputs.) Je corrige et je reviens.

4

Je pense qu'une partie de la solution se trouve ici

Mais je ne suis pas arrivé à faire le lien :s

Avec ceci aussi :

5

Il faut utiliser le user logstash_writer je pense.

Je te copie un message que m'a envoyé @TimV à propos de ton post. Merci Tim !

The problem is that he is trying to use the logstash_system user, but that’s not its intended purpose. The docs (in english, which is not very helpful for him) say

This user has the minimum permissions necessary for the monitoring function, and should not be used for any other purpose - it is specifically not intended for use within a Logstash pipeline.

He’ll need to create a new logstash_writer user, per the top half of the documentation.

6

En fait, j'ai un compte logstash avec le role logstash_writer. Mais les outputs étaient mal configurés.

Et j'ai été induit en erreur par le message d'erreur du début (healthcheck effectué par le compte "utilisateur" ?)

C'est tout bon. je n'ai plus d'erreurs !

Merci pour votre aide.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.