Path.config and config.string mixed

[devopstt]

Hi,

we want to seperate some information in our logstash configurations, to let teams configure the filter and index setting, but not the input ports or ES output settings, like ES hosts, or users.

So we created a config at /etc/logstash/conf.d/fulfillment.conf

input {
    tcp {
        port => 9003
        codec => json_lines
    }
}

filter {
     mutate {
          remove_field => ["@version", "port"]
     }
}

output {
     elasticsearch {
          index => "fulfillment-%{+YYYY.MM.dd}"
    }
}

The pipeline config looks like this (/etc/logstash/pipelines.yml):

- pipeline.id: fulfillment-logger
  path.config: "/etc/logstash/conf.d/fulfillment.conf"
  config.string: |
    output {
      elasticsearch {
        hosts => ["https://1.2.3.4:9200", "https://1.2.3.5:9200", "https://1.2.3.6:9200", "https://1.2.3.7:9200"]
        ssl => true
        ssl_certificate_verification => false
        user => "logstash_fulfillment"
        password => "secure"
        cacert => "/etc/logstash/ca.pem"
      }
    }

The idea is, that a team can manage their LS config with filters and general input and output settings. The LS owner can manage the pipeline config with user, host and certificate information. The ES outshould should be merged from both configs.

Starting LS, the output from ES seems to be fine. But LS will not bind the input tcp port 9003. It looks like I can not mix LS configs and pipeline configs.
How can I seperate the mentioned information? Should I only use one type of configuration file and start using environment variables?

Thanks and regards