Logstash Centralized management pipeline

Elastic Stack Logstash
1

Hi,
I am trying to enable the logstash centralized pipeline management with X-Pack security enabled, but with limited documentation from ELK I am unable to proceed further on how it will look at Kibana side.

Regards
JS

2

Did you follow the instructions in the documentation?

What is not working?

Also, what is your license? You need at least the paid Gold license.

3

Hi,
I am using Enterprise License. I followed this documentation and running the logstash as below.

  1. logstash.yml : x-pack enabled with SSL and
  2. abc.conf: with beat inputs and Elasticsearch as output.
  3. added the abc.conf entry in pipelines.yml
  4. running the logstash as "logstash -f abc.conf
  5. created the pipeline (abc.conf) in "Centralized Logstash Management" section of Stack Management in Kibana.
  6. Made the changes in abc.conf, re-running the pipeline, expecting the changes to reflect in "Centralized Logstash Management"
4

This is not what the documentation is saying.

When using the Centralized Logstash Management you do not use the pipelines.yml anymore, nor you start logstash using the -f parameter, you start logstash and all the pipelines will be synced from your elasticsearch.

I would recommend that you follow these steps on how to configure the Centralized Management Pipeline in logstash.

5

Ok, So I need to create the separate .conf for each logstash and then list in the "xpack.management.pipeline.id".
I think I am all done with the configuration.
Would be great if you can share the recommended way to run the logstash .
Thankyou

6

You do not create configuration files anymore, you will create them through the Kibana interface according to these steps

If you want a pipeline named abc, you will need this line in the logstash.yml file in each one of the logstash instances that you want to run this pipeline:

xpack.management.pipeline.id: ["abc"]

Then you will create in Kibana a pipeline with the ID abc, when your logstash instance starts, it will sync the pipeline configuration.

I don't know what system you are using, but I would recommend that you run logstash as a service, if you are on linux you can use systemd and follow this documentation, if you are on windows you can use nssm and follow this documentation

7

Thanks Leandro much appreciated ! :slightly_smiling_face:

8

Welcome to our community! :smiley:

Please do speak to your Support engineer then, they will be able to help :slight_smile:

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.