Multiple pipelines with distributor pattern

Elastic Stack Logstash
1

I've just started working with multiple pipelines and the distributor pattern. Am I correct in my understanding that I have to copy'n'paste my individual logstash config contents for each pipeline into pipelines.yml in order for it to work properly. I tried the following but the errors in the logs seem to indicate what my question is asking...

- pipeline.id: beats-server
  config.string: |
    input { beats { port => 5200 } }
    output  {
        if [type] == syslog
          {
          pipeline { send_to => syslogs }
          }
            }
- pipeline.id: syslogs
  config.string: |
    input { pipeline
    {
      address => syslogs
        }
    }
  path.config: "/etc/logstash/pipelines/syslogs/*.conf"

If my assumption is correct, are there any plans to allow a config like I posted above?

2

in pipelines.yml :

  • pipeline.id: beats-server
    config.string: |
    input { beats { port => 5200 } }
    output {
    if [type] == syslog
    {
    pipeline { send_to => syslogs }
    }
    }
  • pipeline.id: syslogs
    path.config: "/etc/logstash/pipelines/syslogs/*.conf"

in your .conf :
input { pipeline { address => syslogs } }

3

I've fiddled with this for an hour or so but no luck. When I test my syslog.conf in the proper pipeline directory, I get:

16:34:06.336 [LogStash::Runner] FATAL logstash.runner - The given configuration is invalid. Reason: Couldn't find any input plugin named 'pipeline'. Are you sure this is correct? Trying to load the pipeline input plugin resulted in this error: Problems loading the requested plugin named pipeline of type input. Error: NameError NameError

I used the logstash-plugin command and a 'logstash-input-pipeline' plugin doesn't exist.

Do you actually have this running on a live system or was this an educated guess?

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.