How to understand logstash pipeline-to-pipeline patterns?

Currently I use logstash to operate pipeline.yml, the contents are as follows, according to my understanding, a line of pipeline.id should be a thread, it should be independent, but recently I found that there is a problem with one of the pipeline.id, such as its output endpoint is offline, then the result is that the entire pipeline.yml will not work properly.

- pipeline.id: abp_auditlogs
  path.config: "/etc/logstash/conf.d/abp_auditlogs.conf"
- pipeline.id: lonsid_k8s_dev
  path.config: "/etc/logstash/conf.d/lonsid_k8s_dev.conf"
- pipeline.id: lonsid_k8s_prd
  path.config: "/etc/logstash/conf.d/lonsid_k8s_prd.conf"
- pipeline.id: ack_k8s_dev
  path.config: "/etc/logstash/conf.d/ack_k8s_dev.conf"
- pipeline.id: ack_k8s_prd
  path.config: "/etc/logstash/conf.d/ack_k8s_prd.conf"
- pipeline.id: iiot
  path.config: "/etc/logstash/conf.d/iiot.conf"
- pipeline.id: scheduler
  path.config: "/etc/logstash/conf.d/abp_schedulerlogs.conf"
- pipeline.id: mall_log
  path.config: "/etc/logstash/conf.d/mall_log.conf"
- pipeline.id: iot_masa_pre_prd
  path.config: "/etc/logstash/conf.d/iot_masa_pre_prd.conf"

According to forum searches, I found that logstash has multiple architectural patterns

The documentation uses config.string, but I don't see any difference from my use of path.config, why is it considered isolated mode? Other patterns are similar, I don't understand.

# config/pipelines.yml
- pipeline.id: intake
  config.string: |
    input { beats { port => 5044 } }
    output { pipeline { send_to => [es, http] } }
- pipeline.id: buffered-es
  queue.type: persisted
  config.string: |
    input { pipeline { address => es } }
    output { elasticsearch { } }
- pipeline.id: buffered-http
  queue.type: persisted
  config.string: |
    input { pipeline { address => http } }
    output { http { } }

My purpose is to want logstash to keep running stably.

You can use the output isolator pattern to prevent Logstash from becoming blocked if one of multiple outputs experiences a temporary failure.

That is expected. Read through this thread.

Thank you for your reply, actually the "forum searches result" I mentioned above, what I saw is this thread.

I still don't understand how to configure isolation mode.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.