# Filter multiple logs from same file to different index

**URL:** <https://discuss.elastic.co/t/filter-multiple-logs-from-same-file-to-different-index/234330>\
**Category:** Logstash\
**Created:** [May 26, 2020, 12:22pm UTC](https://discuss.elastic.co/t/filter-multiple-logs-from-same-file-to-different-index/234330 "2020-05-26T12:22:45Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Dariko77](https://avatars.discourse-cdn.com/v4/letter/d/8e7dd6/32.png) [@Dariko77](https://discuss.elastic.co/u/Dariko77)\
**Post date:** [May 26, 2020, 12:22pm UTC](https://discuss.elastic.co/t/filter-multiple-logs-from-same-file-to-different-index/234330/1 "2020-05-26T12:22:45Z")

</div>

Hello,

I have a syslog file that aggregate logs from multiples sources (and I cant separate the logs to different files) and I have a filebeat monitoring that single file that send to logstash.

That file has the following format:

2020-05-22T12:09:18+00:00 10.100.2.137 MESSAGE\_LOG  
2020-05-22T12:09:18+00:00 10.100.1.138 MESSAGE\_LOG  
2020-05-22T12:09:18+00:00 10.100.3.136 MESSAGE\_LOG

In logstash config file I have the following filter (I wanto to parse more fields but I put GREEDYDATA for this example):

```auto
    filter {
            grok {
                    match => { message => "%{TIMESTAMP_ISO8601:timestamp_syslog} %{SYSLOGHOST:syslog_hostname} %{GREEDYDATA:message}"
                    }
            }
    }

```

Now I want to send the logs to different indexes based on %{SYSLOGHOST:syslog\_hostname}.

In output section can I do something like:  
`if [syslog_hostname] == 10.100.2.137{elasticsearch{...}}`

Thank you

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [May 26, 2020, 3:28pm UTC](https://discuss.elastic.co/t/filter-multiple-logs-from-same-file-to-different-index/234330/2 "2020-05-26T15:28:36Z")

</div>

> [@Dariko77](#):
>
> In output section can I do something like:  
> `if [syslog_hostname] == 10.100.2.137{elasticsearch{...}}`

You can but it would be

```
if [syslog_hostname] == "10.100.2.137" {elasticsearch{...}}

```

However, if you do it that way you will have multiple elasticsearch outputs, each maintaining its own connection pool. It might be better to do something like

```
output {
    elasticsearch {
        index => "%{syslog_hostname}"
        ...
    }
}

```

That said, do you really need separate indexes for each host? If you have a large set of hosts that will add a lot of overhead.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 23, 2020, 3:28pm UTC](https://discuss.elastic.co/t/filter-multiple-logs-from-same-file-to-different-index/234330/3 "2020-06-23T15:28:40Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.