# Can I filter a separate set of logs through logstash rather than straight to Elasticsearch?

**URL:** <https://discuss.elastic.co/t/can-i-filter-a-separate-set-of-logs-through-logstash-rather-than-straight-to-elasticsearch/353726>\
**Category:** Logstash\
**Created:** [February 20, 2024, 9:47pm UTC](https://discuss.elastic.co/t/can-i-filter-a-separate-set-of-logs-through-logstash-rather-than-straight-to-elasticsearch/353726 "2024-02-20T21:47:04Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![jreyes25](https://avatars.discourse-cdn.com/v4/letter/j/898d66/32.png) [@jreyes25](https://discuss.elastic.co/u/jreyes25)\
**Post date:** [February 20, 2024, 9:47pm UTC](https://discuss.elastic.co/t/can-i-filter-a-separate-set-of-logs-through-logstash-rather-than-straight-to-elasticsearch/353726/1 "2024-02-20T21:47:04Z")

</div>

Hello,

I currently have Elasticsearch and Kibana installed and configured. I installed fleet-server to manage my elastic-agents and installed elastic-agents to all my hosts and everything is working as should. My elastic-agents are forwarding logs from /var/log/secure, /var/log/messages, and /var/log/audit/audit.log from all my host straight to Elasticsearch.

I do have one particular log located in a separate location on a specific host (i.e. /var/log/dirsrv/slapd/access). I want to send these logs through logstash so I can filter the logs because I only need a few logs that are generated inside it instead of taking in all of the data

I installed logstash (on the same server as elasticsearch and kibana) and created a filter in /etc/logstash.conf.d/access.conf. But I wanted to send only these logs to logstash to get filtered while keeping the other logs (/var/log/secure.... etc....etc) forwarded straight to Elasticsearch. Is that possible?  
Do I just add the output inside the filebeat.yml file located inside the elastic-agent path (/opt/Elastic/Agent/data/elastic-agent/install/filebeat/filebeat.yml) and enable the logstash.yml module or will that just send all logs (on that host machine) through logstash?

---

<div class="post-metadata">

**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\
**Post date:** [February 20, 2024, 9:56pm UTC](https://discuss.elastic.co/t/can-i-filter-a-separate-set-of-logs-through-logstash-rather-than-straight-to-elasticsearch/353726/2 "2024-02-20T21:56:04Z")

</div>

You can add in filebeat.yml per log params.  
field.type = log1  
field.type = log2  
...

Then in LS, add lines:

```auto
if [field][type] =="log1"
.... # code to filter msges
if [field][type] =="log2 " {
drop {} # delete all logs from logs
}

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [February 21, 2024, 12:07am UTC](https://discuss.elastic.co/t/can-i-filter-a-separate-set-of-logs-through-logstash-rather-than-straight-to-elasticsearch/353726/3 "2024-02-21T00:07:59Z")

</div>

A filebeat instance can only have one output, so you would need two filebeat instances. One to send stuff direct to elasticsearch and one to send stuff to logstash. See [this](https://discuss.elastic.co/t/configure-file-beat-to-multiple-output/235471/2) thread.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 20, 2024, 12:08am UTC](https://discuss.elastic.co/t/can-i-filter-a-separate-set-of-logs-through-logstash-rather-than-straight-to-elasticsearch/353726/4 "2024-03-20T00:08:26Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.