# Logstash pipeline with input+ filter to one output and other without filter to different output

**URL:** https://discuss.elastic.co/t/logstash-pipeline-with-input-filter-to-one-output-and-other-without-filter-to-different-output/322350
**Category:** Logstash
**Created:** [January 3, 2023, 8:33am UTC](https://discuss.elastic.co/t/logstash-pipeline-with-input-filter-to-one-output-and-other-without-filter-to-different-output/322350 "2023-01-03T08:33:34Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![shadu88](https://avatars.discourse-cdn.com/v4/letter/s/e19b73/32.png) [@shadu88](https://discuss.elastic.co/u/shadu88)
#### Post date: [January 3, 2023, 8:33am UTC](https://discuss.elastic.co/t/logstash-pipeline-with-input-filter-to-one-output-and-other-without-filter-to-different-output/322350/1 "2023-01-03T08:33:34Z")

</div>

Hello Elkan s  
I wish you a happy new year!!  
Hope all are doings well.

My scenario: collecting logs on port 9600 and adding filter to forward the logs to qradar

I need to add one more output of azure sentinel without filter

When I created another conf file with same input without filter its getting errored

Please suggest solution. I know I need to change something on pipelines.yml  
But not sure though.  
Thank you in advance!!!

---

<div class="post-metadata">

### Author: ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)
#### Post date: [January 3, 2023, 8:52am UTC](https://discuss.elastic.co/t/logstash-pipeline-with-input-filter-to-one-output-and-other-without-filter-to-different-output/322350/2 "2023-01-03T08:52:40Z")

</div>

If you have a single port which accept all traffic, in you case 9600, and you want to forward to multiple locations then use IFs in the output section. Based on value of fields or tags you do redirections.

```auto
output {
if ( [field]=="value1" ) {
    elasticsearch {
    hosts => ["http://host1:9200"]
    index => "index1"
    } 
}
else if ( [field]=="value2" ) {
      microsoft-logstash-output-azure-loganalytics {
        workspace_id => "id"
        workspace_key => "key"
        custom_log_table_name => "tableName"
      }
}

```

Check the [documentation](https://learn.microsoft.com/en-us/azure/sentinel/connect-logstash).

If you have the multiple input ports, for instance 9500 and 9600, then you separate pipelines.

---

<div class="post-metadata">

### Author: ![shadu88](https://avatars.discourse-cdn.com/v4/letter/s/e19b73/32.png) [@shadu88](https://discuss.elastic.co/u/shadu88)
#### Post date: [January 3, 2023, 9:32am UTC](https://discuss.elastic.co/t/logstash-pipeline-with-input-filter-to-one-output-and-other-without-filter-to-different-output/322350/3 "2023-01-03T09:32:33Z")

</div>

Thank you for quick response.

My conf:

Input section: Input on tcp 9600 which accepts raw logs( from heterogenous log sources)

Filter section: set of grok and mutate functions to cleanse

Output : send cleanse( from filter section) data to qradar

New addition : send Raw data without cleansing to sentinel

---

<div class="post-metadata">

### Author: ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)
#### Post date: [January 3, 2023, 10:02am UTC](https://discuss.elastic.co/t/logstash-pipeline-with-input-filter-to-one-output-and-other-without-filter-to-different-output/322350/4 "2023-01-03T10:02:38Z")

</div>

Now is more clear. In your case, the pseudo code is like this:

Add this lines to: config/pipelines.yml

```auto
- pipeline.id: main
  path.config: "/etc/logstash/main.conf"

- pipeline.id: qradar
  path.config: "/etc/logstash/qradar.conf"

```

Create the main.conf file:

```auto
input {
        tcp { port => 9600 }
}
filter {
# for raw messages, do not add any processing code
}
output {
  microsoft-logstash-output-azure-loganalytics {
        workspace_id => "id"
        workspace_key => "key"
        custom_log_table_name => "tableName"
      }
  pipeline { send_to => "qradar" }
}

```

Create the qradar.conf file:

```auto
input {
 pipeline { address => "qradar" }
}
filter {
 # add your existing logic
}
output {
 qradar_connection_string { ....
 }
}

```

Check [the documentation](https://www.elastic.co/guide/en/logstash/current/pipeline-to-pipeline.html)

---

<div class="post-metadata">

### Author: ![shadu88](https://avatars.discourse-cdn.com/v4/letter/s/e19b73/32.png) [@shadu88](https://discuss.elastic.co/u/shadu88)
#### Post date: [January 3, 2023, 10:27am UTC](https://discuss.elastic.co/t/logstash-pipeline-with-input-filter-to-one-output-and-other-without-filter-to-different-output/322350/5 "2023-01-03T10:27:50Z")

</div>

I appreciate it. Shall test in my lab and come back here.

Thank you so much!!!!  
Cheers!!

---

<div class="post-metadata">

### Author: ![shadu88](https://avatars.discourse-cdn.com/v4/letter/s/e19b73/32.png) [@shadu88](https://discuss.elastic.co/u/shadu88)
#### Post date: [January 3, 2023, 2:34pm UTC](https://discuss.elastic.co/t/logstash-pipeline-with-input-filter-to-one-output-and-other-without-filter-to-different-output/322350/6 "2023-01-03T14:34:15Z")

</div>

Hope this works on logstash 7.x?

---

<div class="post-metadata">

### Author: ![shadu88](https://avatars.discourse-cdn.com/v4/letter/s/e19b73/32.png) [@shadu88](https://discuss.elastic.co/u/shadu88)
#### Post date: [January 3, 2023, 4:05pm UTC](https://discuss.elastic.co/t/logstash-pipeline-with-input-filter-to-one-output-and-other-without-filter-to-different-output/322350/7 "2023-01-03T16:05:08Z")

</div>

Test went well using file outputs.  
But I see such ERRORS: "main ERROR Unable to locate appender "

Can be fixed or ignored?

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 31, 2023, 4:05pm UTC](https://discuss.elastic.co/t/logstash-pipeline-with-input-filter-to-one-output-and-other-without-filter-to-different-output/322350/8 "2023-01-31T16:05:29Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
