Unable to merge pipelines in logstash

Elastic Stack Logstash
1

Hi

I am using Logstash version 6.2.4

I am trying to merge pipelines as below:

Master pipeline/Merged Pipeline:

# config/pipelines.yml
- pipeline.id: beats1
  config.string: |
    input { 
	file { 
			path => "/XXXXX/*.txt"
			start_position => "beginning"
			add_field => [log_type, "Beat-2"]
		}
	file { 
			path => "/YYYYY/*.txt"
			start_position => "beginning"
			add_field => [log_type, "Beat-3"]
		}
	}
    output {
        if [log_type] == "Beat-2" {
          pipeline { send_to => beats2 }
        } 
		if [log_type] == "Beat-3" {
          pipeline { send_to => beats3 }
        }
    }

Code in my beats 2 is as below:

- pipeline.id: beats2
  config.string: |
    input { pipeline { address => beats2 } }
    filter {
       # my fileter logic here to parse the data
    }
    output {
      elasticsearch { ## my output server details here }
    }

Code in my beats 3 is as below

- pipeline.id: beats3
  config.string: |
    input { pipeline { address => beats3 } }
    filter {
       # my filter logic here to parse the data
    }
    output {
      elasticsearch { ## my output server details here }
    }

I am deploying these pipelines by going: Kibana Console -> Management -> Logstash -> Pipeline (Using Centralized Pipeline Configuration)

But when I look at logstash logs, I get error "Expected one of #, input, filter, output at line 1, column 1"

This same error is coming against each of these pipelines.

I also checked if pipeline to pipeline communication(distributor pattern) is possible or not on version 6.2.4 but could not find any documentation for that.

Can anyone please help me in resolving this issue.

Can anyone please help me.

Thanks

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.