Logstash to multiple ingest node pipelines

Elastic Stack Logstash
1

I was looking into the forum on how to setup logstash to sent data using more than one ingest node pipelines but could not find nothing related, my objective is to split the load into elastic cloud and my logstash instance to pre process stuff.

I know that there is a pipeline option in the elasticsearch output plugin, but it doesn't accept more than one argument as parameters. After testing the configuration I got this error:

$ /usr/share/logstash/bin/logstash --config.test_and_exit -f 100-beats-default.conf
output {
elasticsearch {
  # This setting must be a string
  # Expected string, got ["kaspersky-drop-false-positive-htm", "kaspersky-extract-hostname-critical-status"]
  pipeline => ["kaspersky-drop-false-positive-htm", "kaspersky-extract-hostname-critical-status"]
  ...
}
}
[FATAL] 2020-10-21 14:00:34.089 [LogStash::Runner] runner - The given configuration is invalid. Reason: Unable to configure plugins: (ConfigurationError) Something is wrong with your configuration.
[ERROR] 2020-10-21 14:04:03.987 [LogStash::Runner] Logstash - java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit

I've tried both without success:

pipeline => "ingest1", "ingest2"
pipeline => ["ingest1", "ingest2"]

Cheers :slight_smile:

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.