# How can filter diiferent logs with different index generated by logstash in kibana **URL:** \ **Category:** Logstash\ **Created:** [May 5, 2020, 2:27am UTC](https://discuss.elastic.co/t/how-can-filter-diiferent-logs-with-different-index-generated-by-logstash-in-kibana/231066 "2020-05-05T02:27:01Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![Ekta](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ekta/32/55987_2.png) [@Ekta](https://discuss.elastic.co/u/Ekta)\ **Post date:** [May 5, 2020, 2:27am UTC](https://discuss.elastic.co/t/how-can-filter-diiferent-logs-with-different-index-generated-by-logstash-in-kibana/231066/1 "2020-05-05T02:27:01Z")
Hi I take log file from different paths and make different config files according to it. Then for output all indexes are created as given in config file but while in kibana it filter with different index it show all logs of all indices? In logstash input plugin is file and output plugin is elasticsearch can anyone help me here? ---
**Author:** ![ptamba](https://avatars.discourse-cdn.com/v4/letter/p/7feea3/32.png) [@ptamba](https://discuss.elastic.co/u/ptamba)\ **Post date:** [May 5, 2020, 3:57am UTC](https://discuss.elastic.co/t/how-can-filter-diiferent-logs-with-different-index-generated-by-logstash-in-kibana/231066/2 "2020-05-05T03:57:39Z")
Kibana works with index pattern. so if you want to see only the logs from specific index, you need to create an index pattern of if. by default , there’s a logstash-\* index pattern in kibana to work with logs generated by logstash. if you have specified a different index name in logstash config, all you need to do is create index pattern that matches those indices. ---
**Author:** ![Ekta](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ekta/32/55987_2.png) [@Ekta](https://discuss.elastic.co/u/Ekta)\ **Post date:** [May 5, 2020, 9:31am UTC](https://discuss.elastic.co/t/how-can-filter-diiferent-logs-with-different-index-generated-by-logstash-in-kibana/231066/3 "2020-05-05T09:31:15Z")
Hi @ptamba, I already create index pattern then I check the log in Discover tab and filter with a specific index name but It shows all logs while selecting anyone index. ---
**Author:** ![ptamba](https://avatars.discourse-cdn.com/v4/letter/p/7feea3/32.png) [@ptamba](https://discuss.elastic.co/u/ptamba)\ **Post date:** [May 5, 2020, 9:33am UTC](https://discuss.elastic.co/t/how-can-filter-diiferent-logs-with-different-index-generated-by-logstash-in-kibana/231066/4 "2020-05-05T09:33:41Z")
you will need to show your logstash output section and the index pattern you use. what happens versus what you expect. otherwise it’s really hard to imagine. ---
**Author:** ![Ekta](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ekta/32/55987_2.png) [@Ekta](https://discuss.elastic.co/u/Ekta)\ **Post date:** [May 5, 2020, 1:26pm UTC](https://discuss.elastic.co/t/how-can-filter-diiferent-logs-with-different-index-generated-by-logstash-in-kibana/231066/5 "2020-05-05T13:26:06Z")
# Sample Logstash configuration for creating a simple # Beats -\> Logstash -\> Elasticsearch pipeline. # i/p File o/p kibana input { file { path =\> "/root/Desktop/mount/LOGS/node01/node01-05-\*.log" start\_position =\> beginning sincedb\_path =\> "/dev/null" } } output { elasticsearch { hosts =\> ["xx.xxx.xxx.xxx:9200"] manage\_template =\> false index =\> "node1\_%{+YYYY.MM}" } } ---
**Author:** ![Ekta](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ekta/32/55987_2.png) [@Ekta](https://discuss.elastic.co/u/Ekta)\ **Post date:** [May 5, 2020, 1:27pm UTC](https://discuss.elastic.co/t/how-can-filter-diiferent-logs-with-different-index-generated-by-logstash-in-kibana/231066/6 "2020-05-05T13:27:19Z")
this is my logstash one config file I created 5 config file for different nodes node01 to 05 ---
**Author:** ![ptamba](https://avatars.discourse-cdn.com/v4/letter/p/7feea3/32.png) [@ptamba](https://discuss.elastic.co/u/ptamba)\ **Post date:** [May 5, 2020, 2:33pm UTC](https://discuss.elastic.co/t/how-can-filter-diiferent-logs-with-different-index-generated-by-logstash-in-kibana/231066/7 "2020-05-05T14:33:43Z")
if you put all config files in the same directory and use that folder as path.config , all your input files will be processed by every single config unless you add fields to differentiate them. per the docs : “If a directory is given, all files in that directory will be concatenated in lexicographical order and then parsed as a single config file” add a tag (or type) to each input and do conditional on the output. that way, each input get processed by correct output. ---
**Author:** ![Ekta](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ekta/32/55987_2.png) [@Ekta](https://discuss.elastic.co/u/Ekta)\ **Post date:** [May 6, 2020, 12:44am UTC](https://discuss.elastic.co/t/how-can-filter-diiferent-logs-with-different-index-generated-by-logstash-in-kibana/231066/8 "2020-05-06T00:44:00Z")
This the path where my all config files saved: /etc/logstash/conf.d/ node01.conf node02.conf node03.conf node04.conf node05.conf I am using below command for run multiple config file sudo /usr/share/logstash/bin/logstash --path.settings /etc/logstash/ --path.data /var/log/ -f /etc/logstash/conf.d & ---
**Author:** ![ptamba](https://avatars.discourse-cdn.com/v4/letter/p/7feea3/32.png) [@ptamba](https://discuss.elastic.co/u/ptamba)\ **Post date:** [May 6, 2020, 4:39am UTC](https://discuss.elastic.co/t/how-can-filter-diiferent-logs-with-different-index-generated-by-logstash-in-kibana/231066/9 "2020-05-06T04:39:00Z")
as i mention, all those configuration files will be concatenated into a large single config. so unless you differentiate events per config file, each files will processed by each config. that's why your index contains events from all sources. you could do for example in (conf1) ```auto input { .... (your existing config) type => "node1" } ``` then on output section ```auto output { if [type] == "node1" { elasticsearch { hosts => ["xx.xxx.xxx.xxx:9200"] manage_template => false index => "node1_%{+YYYY.MM}" } } ``` if you're doing filter processing, ensure that you filter also. uses the same type. use different identifier for different config ---
**Author:** ![Ekta](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ekta/32/55987_2.png) [@Ekta](https://discuss.elastic.co/u/Ekta)\ **Post date:** [May 7, 2020, 1:40pm UTC](https://discuss.elastic.co/t/how-can-filter-diiferent-logs-with-different-index-generated-by-logstash-in-kibana/231066/10 "2020-05-07T13:40:19Z")
Now it's working Thanks a lot @ptamba ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 4, 2020, 1:40pm UTC](https://discuss.elastic.co/t/how-can-filter-diiferent-logs-with-different-index-generated-by-logstash-in-kibana/231066/11 "2020-06-04T13:40:22Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.