# Output logs to multiple index from logstash
**URL:** https://discuss.elastic.co/t/output-logs-to-multiple-index-from-logstash/208991
**Category:** Logstash
**Created:** [November 21, 2019, 11:41pm UTC](https://discuss.elastic.co/t/output-logs-to-multiple-index-from-logstash/208991 "2019-11-21T23:41:29Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@Mehak\_Bhargava](https://discuss.elastic.co/u/Mehak_Bhargava)
#### Post date: [November 21, 2019, 11:41pm UTC](https://discuss.elastic.co/t/output-logs-to-multiple-index-from-logstash/208991/1 "2019-11-21T23:41:29Z")
Hi there,
I am trying to send multiple log files from filebeat tologstash to elasticsearch. In kibana, I would like each log file under a separate index. This is my logstash.conf file:
```auto
# Sample Logstash configuration for creating a simple
# Beats -> Logstash -> Elasticsearch pipeline.
input {
beats {
port => 5044
}
}
filter {
if[type] =="DispatcherApp"{
grok {
match => {"message" => "%{COMBINEDAPACHELOG}"}
}
} else if [type] == "IncidentAgent" {
grok {
match => { "message" => "%{COMBINEDAPACHELOG}" }
}
}else if [type] == "IMMService" {
grok {
match => { "message" => "%{COMBINEDAPACHELOG}" }
}
}
}
output {
elasticsearch {
hosts => ["http://localhost:9200"]
sniffing => true
manage_template => false
index => "web-%{type}"
document_type => "log"
#index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
}
}
```
and filebeat.yml is-
```auto
filebeat.inputs:
-
paths:
- E:\DemoSetup\DispatcherApp\logs\dispatcher-scheduler.log
input_type: log
document_type: DispatcherApp
-
paths:
- E:\DemoSetup\Incident Agent\Logs\Trace.log
input_type: log
document_type: IncidentAgent
-input_type: log
paths:
- E:\DemoSetup\Logs\IMSService\log.txt
input_type: log
document_type: IMMService
#multiline.pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}'
#multiline.negate: true
#multiline.match: after
setup.template.name: "index-%{[beat.version]}"
setup.template.pattern: "index-%{[beat.version]}-*"
output:
logstash:
hosts: ["localhost:5044"]
#index: "index-%{[beat.version]}-%{[fields.type]:other}-%{+yyy.MM.dd}"
```
I just get a new index created called "web-%{type} and all three file logs are collected under it only. I think the type mentioned in filebeat.yml isnt being acknowledge in logstash file due to the if condition only taking message=\> COMMONAPACHE!
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [November 22, 2019, 12:53am UTC](https://discuss.elastic.co/t/output-logs-to-multiple-index-from-logstash/208991/2 "2019-11-22T00:53:42Z")
I do not run filebeat but I know document types are being [removed](https://www.elastic.co/guide/en/elasticsearch/reference/current/removal-of-types.html), so I wouldn't use anything that purports to set type, and thence \_type.
Try using [tags](https://www.elastic.co/guide/en/beats/filebeat/current/add-tags.html). The conditionals and sprintf reference in logstash will look similar.
That said, once each document is tagged with a type, why bother to put them in separate indexes? It is trivial to add a clause to the query to test the tag.
---
### Author:  [@Mehak\_Bhargava](https://discuss.elastic.co/u/Mehak_Bhargava)
#### Post date: [November 22, 2019, 12:59am UTC](https://discuss.elastic.co/t/output-logs-to-multiple-index-from-logstash/208991/3 "2019-11-22T00:59:11Z")
> [@Badger](#):
>
> why bother to put them in separate indexes?
Are you referring to below here?
```auto
setup.template.name: "index-%{[beat.version]}"
setup.template.pattern: "index-%{[beat.version]}-*"
```
> [@Mehak\_Bhargava](#):
>
> I think the type mentioned in filebeat.yml isnt being acknowledge in logstash file due to the if condition only taking message=\> COMMONAPACHE!
Do you agree with this? After I replace tags with document\_types, should I change the filter in logstash.conf to co-relate the tags?
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [November 22, 2019, 2:26pm UTC](https://discuss.elastic.co/t/output-logs-to-multiple-index-from-logstash/208991/4 "2019-11-22T14:26:16Z")
> [@Mehak\_Bhargava](#):
>
> Are you referring to below here?
No, I was referring to
```
index => "web-%{type}"
```
---
### Author:  [@Mehak\_Bhargava](https://discuss.elastic.co/u/Mehak_Bhargava)
#### Post date: [November 22, 2019, 6:40pm UTC](https://discuss.elastic.co/t/output-logs-to-multiple-index-from-logstash/208991/5 "2019-11-22T18:40:23Z")
But then how will logstash know you create index by using the tagged field?
```auto
output {
#if [@metadata][beat] == "filebeat"{
elasticsearch {
hosts => ["http://localhost:9200"]
sniffing => true
manage_template => false
index => "%{[@metadata][type]}-%{+YYYY.MM.dd}"
```
Wont the "%{[@metadata][type] collect the type value in filebeat.yml and create separate index for each log file?
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [November 22, 2019, 6:51pm UTC](https://discuss.elastic.co/t/output-logs-to-multiple-index-from-logstash/208991/6 "2019-11-22T18:51:12Z")
I am not sure, I do not use filebeat.
---
### Author:  [@Mehak\_Bhargava](https://discuss.elastic.co/u/Mehak_Bhargava)
#### Post date: [November 22, 2019, 6:51pm UTC](https://discuss.elastic.co/t/output-logs-to-multiple-index-from-logstash/208991/7 "2019-11-22T18:51:47Z")
Thanks a lot for your help!
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 20, 2019, 6:51pm UTC](https://discuss.elastic.co/t/output-logs-to-multiple-index-from-logstash/208991/8 "2019-12-20T18:51:50Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.