# How to create multiple indexes based on conduction in logstash

**URL:** https://discuss.elastic.co/t/how-to-create-multiple-indexes-based-on-conduction-in-logstash/164020
**Category:** Logstash
**Created:** [January 13, 2019, 5:44am UTC](https://discuss.elastic.co/t/how-to-create-multiple-indexes-based-on-conduction-in-logstash/164020 "2019-01-13T05:44:25Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![Vinit\_Kumar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vinit_kumar/32/74784_2.png) [@Vinit\_Kumar](https://discuss.elastic.co/u/Vinit_Kumar)
#### Post date: [January 13, 2019, 5:44am UTC](https://discuss.elastic.co/t/how-to-create-multiple-indexes-based-on-conduction-in-logstash/164020/1 "2019-01-13T05:44:26Z")

</div>

**Trying to create multiple indexes for elasticsearch in logstash. But my "if conduction" is not creating any single index, without if conduction it is working fine.**

But if I'm using input as file and in logstash without using filebeat then it is working as per my expectation. Can anyone help me for resolution.

```
###filebeat.yml###
=============
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /home/user/vinit/pache/*.log
  fields:
    log_type: apache-log

- type: log
  enabled: true
  paths:
    - /home/user/vinit/boss/*.log
  fields:
    log_type: jboss-log
  fields_under_root: true

###pipeline-conf.conf###
==================

input {
  beats {
    port => 5044
  }
}

filter {
    grok {
           match => { "message" => "^%{IP:CLIENT_IP} (?:-|%{USER:IDEN}) (?:-|%{USER:AUTH}) \[%{HTTPDATE:CREATED_ON}\] \"(?:%{WORD:REQUEST_METHOD} (?:/|%{NOTSPACE:REQUEST})(?: HTTP/%{NUMBER:HTTP_VERSION})?|-)\" %{NUMBER:RESPONSE_CODE} (?:-|%{WORD:BYTES}) (?:-|%{WORD:EXECUTION_TIME})"}
    add_field => {
                "LOG_TYPE" => "api-log"
        }
    overwrite => ["message"]
    }
    grok {
           match => { "message" => "%{HTTPDATE:CREATED_ON}%{NOTSPACE}%{SPACE} (?:-|%{IP:CLIENT_IP})%{SPACE} %{NOTSPACE}(?:-|%{WORD:REQUEST_METHOD}%{SPACE}) (?:-|%{NOTSPACE:REQUEST})(?: HTTP/%{NUMBER:HTTP_VERSION})%{NOTSPACE}(?:-|%{GREEDYDATA:OTHER_INFO}) (?:-|%{NUMBER:RESPONSE_CODE}) (?:-|%{WORD:BYTES}) (?:-|%{WORD:EXECUTION_TIME})"}
    add_field => {
                "LOG_TYPE" => "web-log"
        }
    overwrite => ["message"]
    }
    grok {
           match => { "message" => "%{TIME:CREATED_ON}%{SPACE}\[(?<THREAD>[^\]]+)?\] %{WORD:METHOD}%{SPACE}%{JAVACLASS:CLASS} - (?<MESSAGE_LOG>[^\r\n]+)((\r?\n)(?<extra>(.|\r?\n)+))?"}
    add_field => {
                "LOG_TYPE" => "jboss-log"
        }
    overwrite => ["message"]
    }
}
output {
   if [fields][log_type] == "apache-log"{
     elasticsearch {
     hosts => ["localhost:9200"]
     manage_template => false
     index => "server-logs-apache"
     }
  }
   if [fields][log_type] == "jboss-log" {
     elasticsearch {
     hosts => ["localhost:9200"]
     manage_template => false
     index => "server-logs-jboss"
     }
  }
   stdout { codec => rubydebug }    
}

##Also Tried##
==============
output {
       if "apache-log" in [fields][log_type] {
         elasticsearch {
         hosts => ["localhost:9200"]
         manage_template => false
         index => "server-logs-apache"
         }
      }
       if "jboss-log" in [fields][log_type] {
         elasticsearch {
         hosts => ["localhost:9200"]
         manage_template => false
         index => "server-logs-jboss"
         }
      }
       stdout { codec => rubydebug }    
    }

```

**I'm expecting result as indexes : server-logs-apache, server-logs-jboss but actual output is empty.**

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [January 13, 2019, 12:58pm UTC](https://discuss.elastic.co/t/how-to-create-multiple-indexes-based-on-conduction-in-logstash/164020/2 "2019-01-13T12:58:21Z")

</div>

You have fields\_under\_root set to true, so the events should have [log\_type] set to jboss-log, not [fields][log\_type].

Note that field names are case sensitive, so log\_type and LOG\_TYPE are different fields.

Are you looking at stdout and checking what the actual events look like? If not, I suggest you do so.

---

<div class="post-metadata">

### Author: ![Vinit\_Kumar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vinit_kumar/32/74784_2.png) [@Vinit\_Kumar](https://discuss.elastic.co/u/Vinit_Kumar)
#### Post date: [January 13, 2019, 2:32pm UTC](https://discuss.elastic.co/t/how-to-create-multiple-indexes-based-on-conduction-in-logstash/164020/3 "2019-01-13T14:32:35Z")

</div>

Hi @Badger

Thanks for the reply.

stdout is showing \_type is doc and log\_type is still not working and the weird thing is I'm not getting any error.

```
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /home/user/vinit/pache/*.log
  fields:
    log_type: apache-log

- type: log
  enabled: true
  paths:
    - /home/user/vinit/boss/*.log
  fields:
    log_type: jboss-log
  fields_under_root: true
```

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [January 13, 2019, 3:13pm UTC](https://discuss.elastic.co/t/how-to-create-multiple-indexes-based-on-conduction-in-logstash/164020/4 "2019-01-13T15:13:47Z")

</div>

In stdout, do you have [log\_type] or do you have [fields][log\_type]?

---

<div class="post-metadata">

### Author: ![Vinit\_Kumar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vinit_kumar/32/74784_2.png) [@Vinit\_Kumar](https://discuss.elastic.co/u/Vinit_Kumar)
#### Post date: [January 13, 2019, 3:20pm UTC](https://discuss.elastic.co/t/how-to-create-multiple-indexes-based-on-conduction-in-logstash/164020/5 "2019-01-13T15:20:11Z")

</div>

I ran this script to check data  
`curl -XGET 'http://localhost:9200/filebeat-*/_search?pretty'` and I didn't get neither [log\_type] nor [fields][log\_type].

I am getting old data which I was created first time this is another problem I'm facing.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 10, 2019, 3:20pm UTC](https://discuss.elastic.co/t/how-to-create-multiple-indexes-based-on-conduction-in-logstash/164020/6 "2019-02-10T15:20:31Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.