# Multiple index in elasticsearch from filebeat

**URL:** https://discuss.elastic.co/t/multiple-index-in-elasticsearch-from-filebeat/92154
**Category:** Beats
**Tags:** filebeat
**Created:** [July 6, 2017, 6:10pm UTC](https://discuss.elastic.co/t/multiple-index-in-elasticsearch-from-filebeat/92154 "2017-07-06T18:10:26Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![Roque\_Moyano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/roque_moyano/32/116329_2.png) [@Roque\_Moyano](https://discuss.elastic.co/u/Roque_Moyano)
#### Post date: [July 6, 2017, 6:10pm UTC](https://discuss.elastic.co/t/multiple-index-in-elasticsearch-from-filebeat/92154/1 "2017-07-06T18:10:27Z")

</div>

Hi , I want to create a two outputs filters in logstash , that depends on the "type" field sent from filebeat:

```
     filebeat:
       prospectors:
        -
          paths:
                  - /var/log/app.log
               
          input_type: log
          document_type: app
        
        paths:
                  - /var/log/stuff.log
               
          input_type: log
          document_type: stuff

```

I have in logstash two filterss:

filterapp.conf

```
  filter {
    if[type] == "app" {
    
    .......         
}

```

filterfoo.conf

```
  filter {
    if[type] == "stuff " {
    
    .......         
}

```

output.conf

```
output {
  elasticsearch {
    hosts => ["localhost:9200"]
    sniffing => true
    manage_template => false
    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
    document_type => "%{[@metadata][type]}"
  }
}

```

I would like to create 2 outputs with 2 indexes

app-\*  
stuff-\*

How Can I tell to output {} to send the logs to app index or stuff index?

---

<div class="post-metadata">

### Author: ![Roque\_Moyano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/roque_moyano/32/116329_2.png) [@Roque\_Moyano](https://discuss.elastic.co/u/Roque_Moyano)
#### Post date: [July 6, 2017, 7:21pm UTC](https://discuss.elastic.co/t/multiple-index-in-elasticsearch-from-filebeat/92154/2 "2017-07-06T19:21:04Z")

</div>

I dont know if this is the correct way , but it seems to be working:

output {

if [type] == "git" {

elasticsearch {  
hosts =\> ["localhost:9200"]  
sniffing =\> true  
manage\_template =\> false  
index =\> "stash-%{+YYYY.MM.dd}"  
document\_type =\> "%{[@metadata][type]}"  
}  
}

else  
{

elasticsearch {  
hosts =\> ["localhost:9200"]  
sniffing =\> true  
manage\_template =\> false  
index =\> "%{[@metadata][beat]}-%{+YYYY.MM.dd}"  
document\_type =\> "%{[@metadata][type]}"  
}  
}

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [July 6, 2017, 10:32pm UTC](https://discuss.elastic.co/t/multiple-index-in-elasticsearch-from-filebeat/92154/3 "2017-07-06T22:32:56Z")

</div>

That is the best way 🙂

---

<div class="post-metadata">

### Author: ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)
#### Post date: [July 7, 2017, 2:26pm UTC](https://discuss.elastic.co/t/multiple-index-in-elasticsearch-from-filebeat/92154/4 "2017-07-07T14:26:05Z")

</div>

yeah, seems to work.

You can use the `mutate` filter to overwrite `@metadata.type` with another value in the filter section. Like:

```auto
filter {
  if [type] == "git" {
    mutate ...
  }
}

elasticsearch {
  hosts => ["localhost:9200"]
  sniffing => true
  manage_template => false
  index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
  document_type => "%{[@metadata][type]}"
  }
}

```

Using `@metadata` in logstash is for 'private' use in your script. You can set/remove any fields at will. Logstash will remove `@metadata` when serializing the event to JSON.

Instead of overwriting '@metadata.beat', you can also create `@metadata.index` in the filter and use `index => "{[@metadata][index]}"`.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 4, 2017, 2:26pm UTC](https://discuss.elastic.co/t/multiple-index-in-elasticsearch-from-filebeat/92154/5 "2017-08-04T14:26:13Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
