Configuring Centralized pipeline management for logstash

Elastic Stack Logstash
1

Hi

I am trying to configure centralized pipeline management for logstash (Version 6.6.0)

By referring to ELK documentation, I have made below changes:

In logstash.yml

xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.url: ["http://localhost:9200"]
xpack.monitoring.elasticsearch.sniffing: true
xpack.monitoring.collection.interval: 10s
xpack.monitoring.collection.pipeline.details.enabled: true

xpack.management.enabled: true
xpack.management.pipeline.id: ["beats1","beats2","beats3","beats4","beats5"]
xpack.management.elasticsearch.url: ["http://localhost:9200"]
xpack.management.elasticsearch.sniffing: true
xpack.management.logstash.poll_interval: 5s

In pipelines.yml

  • pipeline.id: "beats1"
    pipeline.workers: 3
    pipeline.batch.size: 2500
    path.config: "C:/ELK/logstash-6.6.0/config/beats1.conf"
  • pipeline.id: "beats2"
    pipeline.workers: 3
    pipeline.batch.size: 2500
    path.config: "C:/ELK/logstash-6.6.0/config/beats2.conf"
  • pipeline.id: "beats3"
    pipeline.workers: 3
    pipeline.batch.size: 2500
    path.config: "C:/ELK/logstash-6.6.0/config/beats3.conf"
  • pipeline.id: "beats4"
    pipeline.workers: 3
    pipeline.batch.size: 2500
    path.config: "C:/ELK/logstash-6.6.0/config/beats4.conf"
  • pipeline.id: "beats5"
    pipeline.workers: 3
    pipeline.batch.size: 2500
    path.config: "C:/ELK/logstash-6.6.0/config/beats5.conf"

In elasticsearch.yml
xpack.monitoring.collection.enabled: true

But now when I go to command prompt and start logstash, I get below error:

PS C:\ELK\logstash-6.6.0\bin> .\logstash
Sending Logstash logs to C:/ELK/logstash-6.6.0/logs which is now configured via log4j2.properties
[2019-02-13T15:58:31,520][INFO ][logstash.configmanagement.bootstrapcheck] Using Elasticsearch as config store {:pipelin
e_id=>["beats1", "beats2", "beats3", "beats4", "beats5"], :poll_interval=>"5000000000ns"}
[2019-02-13T15:58:31,566][FATAL][logstash.runner ] An unexpected error occurred! {:error=>#<ArgumentError: You
must set the password using the "xpack.management.elasticsearch.password" in logstash.yml>, :backtrace=>["C:/ELK/logstas
h-6.6.0/x-pack/lib/config_management/elasticsearch_source.rb:39:in initialize'", "C:/ELK/logstash-6.6.0/x-pack/lib/conf ig_management/hooks.rb:41:in after_bootstrap_checks'", "org/logstash/execution/EventDispatcherExt.java:69:in execute'" , "C:/ELK/logstash-6.6.0/logstash-core/lib/logstash/runner.rb:293:in execute'", "C:/ELK/logstash-6.6.0/vendor/bundle/jr
uby/2.3.0/gems/clamp-0.6.5/lib/clamp/command.rb:67:in run'", "C:/ELK/logstash-6.6.0/logstash-core/lib/logstash/runner.r b:237:in run'", "C:/ELK/logstash-6.6.0/vendor/bundle/jruby/2.3.0/gems/clamp-0.6.5/lib/clamp/command.rb:132:in run'", " C:\\ELK\\logstash-6.6.0\\lib\\bootstrap\\environment.rb:73:in '"]}
[2019-02-13T15:58:31,582][ERROR][org.logstash.Logstash ] java.lang.IllegalStateException: Logstash stopped processing
because of an error: (SystemExit) exit
PS C:\ELK\logstash-6.6.0\bin>

I have not set up user name and password but still I am getting X-pack security error.

Can anyone please help.

Thanks

2

According to the docs, Centralized pipeline management requires security to be configured.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.