# How to configure different indexes in logstash

**URL:** <https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665>\
**Category:** Logstash\
**Created:** [August 23, 2016, 9:55am UTC](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665 "2016-08-23T09:55:53Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![rocky4bmw](https://avatars.discourse-cdn.com/v4/letter/r/6bbea6/32.png) [@rocky4bmw](https://discuss.elastic.co/u/rocky4bmw)\
**Post date:** [August 23, 2016, 9:55am UTC](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665/1 "2016-08-23T09:55:53Z")

</div>

Hi Team,

Can anyone help me in confugiring multiple indexes so that logs are shipped to different indices based on the environment type(PROD,SIT & DEV). Currently my stepup is working with default filebeat-\* index

Logstash configuration  
input {  
beats {  
port =\> "5044"  
ssl =\> true  
ssl\_certificate =\> "/etc/pki/tls/certs/logstash-forwarder.crt"  
ssl\_key =\> "/etc/pki/tls/private/logstash-forwarder.key"  
}  
}  
filter {  
if [type] == "syslog" {  
grok {  
match =\> { "message" =\> "%{SYSLOGTIMESTAMP:syslog\_timestamp} %{SYSLOGHOST:syslog\_hostname} %{DATA:syslog\_program}(?:[%{POSINT:syslog\_pid}])?: %{GREEDYDATA:syslog\_message}" }  
add\_field =\> ["received\_at", "%{@timestamp}"]  
add\_field =\> ["received\_from", "%{host}"]  
}  
syslog\_pri { }  
date {  
match =\> ["syslog\_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"]  
}  
}  
}  
output {  
elasticsearch {  
hosts =\> ["localhost:9200"]  
user =\> "xxx"  
password =\> "xxx"  
ssl =\> true  
ssl\_certificate\_verification =\> true  
index =\> "%{[@metadata][beat]}-%{+YYYY.MM.dd}"  
document\_type =\> "%{[@metadata][type]}"  
}  
}

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 23, 2016, 10:56am UTC](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665/2 "2016-08-23T10:56:23Z")

</div>

This should help:

> <https://stackoverflow.com/questions/27146032/make-logstash-add-different-inputs-to-different-indices/27147688#27147688>

---

<div class="post-metadata">

**Author:** ![rocky4bmw](https://avatars.discourse-cdn.com/v4/letter/r/6bbea6/32.png) [@rocky4bmw](https://discuss.elastic.co/u/rocky4bmw)\
**Post date:** [August 24, 2016, 12:42am UTC](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665/3 "2016-08-24T00:42:30Z")

</div>

@magnusbaeck Thanks for your response.

I am using filebeat I have define path of logfile in paths:

- /opt/example/\*.log  
input\_type: log

I want to look all the errormessages and error message details so I tried to index both messages

2016-Aug-21 08:11:41 646;ERROR ;Thread-21;YFS10003 ;[1471939901646] YFS:Invalid Order ; [system]; IntegrationAdapter

2016-Aug-21 08:11:41 647;ERRORDTL;Thread-21;YFS10003 ;[1471939901646]\<?xml version="1.0" encoding="UTF-8"?\>

I followed your steps and configured logstash

input {  
beats {  
port =\> "5044"  
ssl =\> true  
ssl\_certificate =\> "/etc/pki/tls/certs/logstash-forwarder.crt"  
ssl\_key =\> "/etc/pki/tls/private/logstash-forwarder.key"  
type =\> "ERROR"  
}  
beats {  
port =\> "5044"  
ssl =\> true  
ssl\_certificate =\> "/etc/pki/tls/certs/logstash-forwarder.crt"  
ssl\_key =\> "/etc/pki/tls/private/logstash-forwarder.key"  
type =\> "ERRORDTL"  
}  
path =\> ["/opt/example/\*.log"]  
type =\> "syslog"  
}  
}

filter {  
if [type] == "syslog" {  
grok {  
match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:time} %{LOGLEVEL:ERR?} [%{NUMBER:thread}] %{JAVACLASS:class} - %{GREEDYDATA:msg}" }  
}  
date {  
match =\> ["syslog\_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"]  
}  
}  
}  
output{  
if [type] == " ERROR " {  
elasticsearch {  
hosts =\> ["localhost:9200"]  
user =\> "xxx"  
password =\> "xxx"  
ssl =\> true  
ssl\_certificate\_verification =\> true  
index =\> " ERROR"  
}  
} else {  
elasticsearch {  
hosts =\> ["localhost:9200"]  
user =\> "xx"  
password =\> "xxx"  
ssl =\> true  
ssl\_certificate\_verification =\> true  
index =\> " ERRORDTL"  
}  
}  
}

Currently I am not getting any logs to logstash. Can you please let me know where could be the error.

I am new to this grokfilters and multiple indexing.

As I mentioned in firstpost everything works fine with default filebeat-\* index.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 24, 2016, 6:08am UTC](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665/4 "2016-08-24T06:08:00Z")

</div>

> input {  
> beats {  
> port =\> "5044"  
> ssl =\> true  
> ssl\_certificate =\> "/etc/pki/tls/certs/logstash-forwarder.crt"  
> ssl\_key =\> "/etc/pki/tls/private/logstash-forwarder.key"  
> type =\> "ERROR"  
> }  
> beats {  
> port =\> "5044"  
> ssl =\> true  
> ssl\_certificate =\> "/etc/pki/tls/certs/logstash-forwarder.crt"  
> ssl\_key =\> "/etc/pki/tls/private/logstash-forwarder.key"  
> type =\> "ERRORDTL"  
> }  
> path =\> ["/opt/example/\*.log"]  
> type =\> "syslog"  
> }  
> }

Two problems make this configuration invalid:

- You can't have to beats inputs that listen to the same port.
- The `path` and `type` options belong inside a `file` block. Is this what your config looks like or is it a copy/paste mistake?

> Currently I am not getting any logs to logstash.

Not any logs to Logstash or not any logs to Elasticsearch?

> ```
> if [type] == " ERROR " {
> 
> ```

Why do you have spaces on both sides of "ERROR"?

> ```
> index => " ERROR"
> index => " ERRORDTL"
> 
> ```

Why do you have leading spaces in the index names?

---

<div class="post-metadata">

**Author:** ![rocky4bmw](https://avatars.discourse-cdn.com/v4/letter/r/6bbea6/32.png) [@rocky4bmw](https://discuss.elastic.co/u/rocky4bmw)\
**Post date:** [August 25, 2016, 6:41am UTC](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665/5 "2016-08-25T06:41:56Z")

</div>

@magnusbaeck Thanks for your response

Actually I am looking for logs from dev servers gone to filebeat-dev index and sit server logs to filebeat-sit index.

How can I achieve this from existing setup. I have shared you field details and current logstash configuration where all logs of different environments shown under filebeat- indexin kibana

---

<div class="post-metadata">

**Author:** ![rocky4bmw](https://avatars.discourse-cdn.com/v4/letter/r/6bbea6/32.png) [@rocky4bmw](https://discuss.elastic.co/u/rocky4bmw)\
**Post date:** [August 25, 2016, 6:42am UTC](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665/6 "2016-08-25T06:42:12Z")

</div>

**_fields of filebeats_** \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*

curl -XGET "[https://localhost:9200/filebeat-\*/\_search?pretty](https://localhost:9200/filebeat-*/_search?pretty)" -u logstash  
Enter host password for user 'logstash':  
{  
"took" : 36,  
"timed\_out" : false,  
"\_shards" : {  
"total" : 100,  
"successful" : 100,  
"failed" : 0  
},  
"hits" : {  
"total" : 25882438,  
"max\_score" : 1.0,  
"hits" : [ {  
"\_index" : "filebeat-2016.08.06",  
"\_type" : "log",  
"\_id" : "AVZjRgYRFjxIs5i8r3d4",  
"\_score" : 1.0,  
"\_source" : {  
"message" : "16:07:20,279 DEBUG : # multipath.conf written by anaconda",  
"@version" : "1",  
"@timestamp" : "2016-08-06T12:41:15.576Z",  
"count" : 1,  
"offset" : 4222,  
"type" : "log",  
"input\_type" : "log",  
"source" : "/var/log/anaconda.storage.log",  
"fields" : null,  
"beat" : {  
"hostname" : "[dev.example.com](http://dev.example.com)",  
"name" : "[dev.example.com](http://dev.example.com)"  
},  
"host" : "[dev.example.com](http://dev.example.com)",  
"tags" : ["beats\_input\_codec\_plain\_applied"]  
}  
},

---

<div class="post-metadata">

**Author:** ![rocky4bmw](https://avatars.discourse-cdn.com/v4/letter/r/6bbea6/32.png) [@rocky4bmw](https://discuss.elastic.co/u/rocky4bmw)\
**Post date:** [August 25, 2016, 6:42am UTC](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665/7 "2016-08-25T06:42:46Z")

</div>

\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*filbeat.json template which I have installed \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*

{  
"mappings": {  
"_default_": {  
"\_all": {  
"enabled": true,  
"norms": {  
"enabled": false  
}  
},  
"dynamic\_templates": [  
{  
"template1": {  
"mapping": {  
"doc\_values": true,  
"ignore\_above": 1024,  
"index": "not\_analyzed",  
"type": "{dynamic\_type}"  
},  
"match": "_"  
}  
}  
],  
"properties": {  
"@timestamp": {  
"type": "date"  
},  
"message": {  
"type": "string",  
"index": "analyzed"  
},  
"offset": {  
"type": "long",  
"doc\_values": "true"  
}  
}  
}  
},  
"settings": {  
"index.refresh\_interval": "5s"  
},  
"template": "filebeat-_"  
}

\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*current logstash configuration \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*

input {  
beats {  
port =\> "5044"  
ssl =\> true  
ssl\_certificate =\> "/etc/pki/tls/certs/logstash-forwarder.crt"  
ssl\_key =\> "/etc/pki/tls/private/logstash-forwarder.key"  
}  
}  
filter {  
if [type] == "syslog" {  
grok {  
match =\> { "message" =\> "%{SYSLOGTIMESTAMP:syslog\_timestamp} %{SYSLOGHOST:syslog\_hostname} %{DATA:syslog\_program}(?:[%{POSINT:syslog\_pid}])?: %{GREEDYDATA:syslog\_message}" }  
add\_field =\> ["received\_at", "%{@timestamp}"]  
add\_field =\> ["received\_from", "%{host}"]  
}  
syslog\_pri { }  
date {  
match =\> ["syslog\_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"]  
}  
}  
}  
output {  
elasticsearch {  
hosts =\> ["localhost:9200"]  
user =\> "xxx"  
password =\> "xxx"  
ssl =\> true  
index =\> "%{[@metadata][beat]}-%{+YYYY.MM.dd}"  
document\_type =\> "%{[@metadata][type]}"  
}  
}  
~  
~

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 25, 2016, 7:22am UTC](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665/8 "2016-08-25T07:22:50Z")

</div>

> Actually I am looking for logs from dev servers gone to filebeat-dev index and sit server logs to filebeat-sit index.

I believe that's covered by the StackOverflow post I referred you to earlier. If not, please ask a specific question.

---

<div class="post-metadata">

**Author:** ![rocky4bmw](https://avatars.discourse-cdn.com/v4/letter/r/6bbea6/32.png) [@rocky4bmw](https://discuss.elastic.co/u/rocky4bmw)\
**Post date:** [August 25, 2016, 9:50am UTC](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665/9 "2016-08-25T09:50:16Z")

</div>

@magnusbaeck how can get "type" field here as per your post. I am using filebeats on client servers and JSON template I installed on master. Already shared you the fileds in old post.

Should I give logfilepath again if I am using beats?

Request you to help on this

**POST** \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*  
input {  
udp {  
...  
type =\> "foo"  
}  
file {  
...  
type =\> "bar"  
}  
}

output {  
if [type] == "foo" {  
elasticsearch {  
...  
index =\> "foo-index"  
}  
} else {  
elasticsearch {  
...  
index =\> "bar-index"  
}  
}  
}

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 25, 2016, 10:55am UTC](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665/10 "2016-08-25T10:55:20Z")

</div>

You don't have to use the `type` field, you can use any field you like. Maybe the hostname in the `host` field can be used? That's up to you.

---

<div class="post-metadata">

**Author:** ![rocky4bmw](https://avatars.discourse-cdn.com/v4/letter/r/6bbea6/32.png) [@rocky4bmw](https://discuss.elastic.co/u/rocky4bmw)\
**Post date:** [August 26, 2016, 12:33am UTC](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665/11 "2016-08-26T00:33:59Z")

</div>

@magnusbaeck thanks for your response.

I have configured my logstash output based on hostname but it doesnot reflect in Kibana.

I tried to search for index filbeatdev or filebeatsit but nothing is shown like that.

Not able to see any error messages in logstash  
final message says  
{:timestamp=\>"2016-08-25T19:29:35.213000-0500", :message=\>"Pipeline main started"}

\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*configuration is as below  
multiple indexing

input {  
beats {  
port =\> "5044"  
ssl =\> true  
ssl\_certificate =\> "/etc/pki/tls/certs/logstash-forwarder.crt"  
ssl\_key =\> "/etc/pki/tls/private/logstash-forwarder.key"  
}  
}  
filter {  
if [type] == "syslog" {  
grok {  
match =\> { "message" =\> "%{SYSLOGTIMESTAMP:syslog\_timestamp} %{SYSLOGHOST:syslog\_hostname} %{DATA:syslog\_program}(?:[%{POSINT:syslog\_pid}])?: %{GREEDYDATA:syslog\_message}" }  
add\_field =\> ["received\_at", "%{@timestamp}"]  
add\_field =\> ["received\_from", "%{host}"]  
}  
syslog\_pri { }  
date {  
match =\> ["syslog\_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"]  
}  
}  
}

output{  
if [host] == ["[example1.sit.com](http://example1.sit.com)","[example2.sit.com](http://example2.sit.com)"] {  
elasticsearch {  
hosts =\> ["localhost:9200"]  
user =\> "xxx"  
password =\> "xxx"  
ssl =\> true  
ssl\_certificate\_verification =\> true  
truststore =\> "/xxxx"  
truststore\_password =\> "xxxx"  
index =\> "%{[@metadata][beatsit]}"  
document\_type =\> "%{[@metadata][type]}"  
}  
} else {  
elasticsearch {  
hosts =\> ["localhost:9200"]  
user =\> "xxx"  
password =\> "xxx"  
ssl =\> true  
ssl\_certificate\_verification =\> true  
truststore =\> "/xxxx"  
truststore\_password =\> "xxxx"  
index =\> "%{[@metadata][beatdev]}"  
document\_type =\> "%{[@metadata][type]}"  
}  
}  
}

---

<div class="post-metadata">

**Author:** ![rocky4bmw](https://avatars.discourse-cdn.com/v4/letter/r/6bbea6/32.png) [@rocky4bmw](https://discuss.elastic.co/u/rocky4bmw)\
**Post date:** [August 26, 2016, 5:05am UTC](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665/12 "2016-08-26T05:05:29Z")

</div>

I see below errors in logstash now

:message=\>"Failed action. ", :status=\>400, :action=\>["index", {:\_id=\>nil, :\_index=\>"%{[@metadata][beatdev]}--2016.08.26", :\_type=\>"filesystem", :\_routing=\>nil}, #\<LogStash::Event:0x2274705f @metadata\_accessors=#\<LogStash::Util::Accessors:0x1fb6eb66 @store={"type"=\>"filesystem", "beat"=\>"topbeat"}, @lut={"[type]"=\>[{"type"=\>"filesystem", "beat"=\>"topbeat"}, "type"], "[beatdev]"=\>[{"type"=\>"filesystem", "beat"=\>"topbeat"}, "beatdev"]}\>, @cancelled=false, @data={"@timestamp"=\>"2016-08-26T04:49:59.610Z", "type"=\>"filesystem", "fs"=\>{"device\_name"=\>"none", "total"=\>0, "used"=\>0, "used\_p"=\>0, "free"=\>0, "avail"=\>0, "files"=\>0, "free\_files"=\>0, "mount\_point"=\>"/proc/sys/fs/binfmt\_misc"}, "count"=\>1, "beat"=\>{"hostname"=\>"[examplebuild.build.com](http://examplebuild.build.com)", "name"=\>"[examplebuild.build.com](http://examplebuild.build.com)"}, "@version"=\>"1", "host"=\>"[examplebuild.build.com](http://examplebuild.build.com)", "tags"=\>["beats\_input\_raw\_event"]}, @metadata={"type"=\>"filesystem", "beat"=\>"topbeat"}, @accessors=#\<LogStash::Util::Accessors:0x16b84b51 @store={"@timestamp"=\>"2016-08-26T04:49:59.610Z", "type"=\>"filesystem", "fs"=\>{"device\_name"=\>"none", "total"=\>0, "used"=\>0, "used\_p"=\>0, "free"=\>0, "avail"=\>0, "files"=\>0, "free\_files"=\>0, "mount\_point"=\>"/proc/sys/fs/binfmt\_misc"}, "count"=\>1, "beat"=\>{"hostname"=\>"[examplebuild.build.com](http://examplebuild.build.com)", "name"=\>"[examplebuild.build.com](http://examplebuild.build.com)"}, "@version"=\>"1", "host"=\>"[examplebuild.build.com](http://examplebuild.build.com)", "tags"=\>["beats\_input\_raw\_event"]}, @lut={"[beat][hostname]"=\>[{"hostname"=\>"[examplebuild.build.com](http://examplebuild.build.com)", "name"=\>"[examplebuild.build.com](http://examplebuild.build.com)"}, "hostname"], "host"=\>[{"@timestamp"=\>"2016-08-26T04:49:59.610Z", "type"=\>"filesystem", "fs"=\>{"device\_name"=\>"none", "total"=\>0, "used"=\>0, "used\_p"=\>0, "free"=\>0, "avail"=\>0, "files"=\>0, "free\_files"=\>0, "mount\_point"=\>"/proc/sys/fs/binfmt\_misc"}, "count"=\>1, "beat"=\>{"hostname"=\>"[examplebuild.build.com](http://examplebuild.build.com)", "name"=\>"[examplebuild.build.com](http://examplebuild.build.com)"}, "@version"=\>"1", "host"=\>"[examplebuild.build.com](http://examplebuild.build.com)", "tags"=\>["beats\_input\_raw\_event"]}, "host"], "tags"=\>[{"@timestamp"=\>"2016-08-26T04:49:59.610Z", "type"=\>"filesystem", "fs"=\>{"device\_name"=\>"none", "total"=\>0, "used"=\>0, "used\_p"=\>0, "free"=\>0, "avail"=\>0, "files"=\>0, "free\_files"=\>0, "mount\_point"=\>"/proc/sys/fs/binfmt\_misc"}, "count"=\>1, "beat"=\>{"hostname"=\>"[examplebuild.build.com](http://examplebuild.build.com)", "name"=\>"[examplebuild.build.com](http://examplebuild.build.com)"}, "@version"=\>"1", "host"=\>"[examplebuild.build.com](http://examplebuild.build.com)", "tags"=\>["beats\_input\_raw\_event"]}, "tags"], "[type]"=\>[{"@timestamp"=\>"2016-08-26T04:49:59.610Z", "type"=\>"filesystem", "fs"=\>{"device\_name"=\>"none", "total"=\>0, "used"=\>0, "used\_p"=\>0, "free"=\>0, "avail"=\>0, "files"=\>0, "free\_files"=\>0, "mount\_point"=\>"/proc/sys/fs/binfmt\_misc"}, "count"=\>1, "beat"=\>{"hostname"=\>"[examplebuild.build.com](http://examplebuild.build.com)", "name"=\>"[examplebuild.build.com](http://examplebuild.build.com)"}, "@version"=\>"1", "host"=\>"[examplebuild.build.com](http://examplebuild.build.com)", "tags"=\>["beats\_input\_raw\_event"]}, "type"], "[host]"=\>[{"@timestamp"=\>"2016-08-26T04:49:59.610Z", "type"=\>"filesystem", "fs"=\>{"device\_name"=\>"none", "total"=\>0, "used"=\>0, "used\_p"=\>0, "free"=\>0, "avail"=\>0, "files"=\>0, "free\_files"=\>0, "mount\_point"=\>"/proc/sys/fs/binfmt\_misc"}, "count"=\>1, "beat"=\>{"hostname"=\>"[examplebuild.build.com](http://examplebuild.build.com)", "name"=\>"[examplebuild.build.com](http://examplebuild.build.com)"}, "@version"=\>"1", "host"=\>"[examplebuild.build.com](http://examplebuild.build.com)", "tags"=\>["beats\_input\_raw\_event"]}, "host"]}\>\>], :response=\>{"create"=\>{"\_index"=\>"%{[@metadata][beatdev]}--2016.08.26", "\_type"=\>"filesystem", "\_id"=\>"AVbFLu-gVOSyDPZ\_PtOh", "status"=\>400, "error"=\>{"type"=\>"illegal\_argument\_exception", "reason"=\>"mapper [fs.used\_p] of different type, current\_type [double], merged\_type [long]"}}}, :level=\>:warn}

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 26, 2016, 7:25am UTC](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665/13 "2016-08-26T07:25:31Z")

</div>

> index =\> "%{[@metadata][beatsit]}"

What is this line supposed to mean? Do you have a `[@metadata][beatsit]` field in your events? What name do you want to use?

---

<div class="post-metadata">

**Author:** ![rocky4bmw](https://avatars.discourse-cdn.com/v4/letter/r/6bbea6/32.png) [@rocky4bmw](https://discuss.elastic.co/u/rocky4bmw)\
**Post date:** [August 26, 2016, 8:36am UTC](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665/14 "2016-08-26T08:36:28Z")

</div>

@magnusbaeck  
logfile is as below

2016-Aug-24 22:14:31 400;WARN ;RMI TCP Connection(37948)-10.136.232.50; ;Clearing cache. Number cached=0 ; [system]; YFS\_Additional\_AttributeDBCacheHome @version:1 source:/opt/apps/Sterling94/Foundation/logs/sci\_.log type:log input\_type:log beat.hostname:examplesit.sit.com [beat.name](http://beat.name):examplesit.sit.com host:examplesit.sit.com tags:beats\_input\_codec\_plain\_applied @timestamp:August 25th 2016, 11:13:32.846 offset:2,171,741 count:1 fields: - \_id:AVbAjssd\_SBZg0ZCoQ0R \_type:log \_index:filebeat-2016.08.25 \_score: -

previously with single index it will give filebeat-\* as index

configuration as below  
output {  
elasticsearch {  
hosts =\> ["localhost:9200"]  
user =\> "xxx"  
password =\> "xxx"  
ssl =\> true  
ssl\_certificate\_verification =\> true  
truststore =\> "xxx"  
truststore\_password =\> "xxx"  
index =\> "%{[@metadata][beat]}-%{+YYYY.MM.dd}"  
document\_type =\> "%{[@metadata][type]}"  
}

Can you please help me here what I am doing wrong as I feel that I am close to crack multiple indexing but something is missing

Should I use different grok filter?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 26, 2016, 8:49am UTC](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665/15 "2016-08-26T08:49:28Z")

</div>

Please answer my questions. What index name do you want to use for the events from example?.sit.com?

Also,

> ```
> if [host] == ["example1.sit.com","example2.sit.com"] {
> 
> ```

doesn't work the way you think. This is what you want:

```
if [host] in ["example1.sit.com", "example2.sit.com"] {

```

---

<div class="post-metadata">

**Author:** ![rocky4bmw](https://avatars.discourse-cdn.com/v4/letter/r/6bbea6/32.png) [@rocky4bmw](https://discuss.elastic.co/u/rocky4bmw)\
**Post date:** [August 26, 2016, 8:52am UTC](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665/16 "2016-08-26T08:52:00Z")

</div>

I want to use "beats\_sit" as indexname for sit environment (example1 and example2)  
and other servers as "beats\_dev" as index name.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 26, 2016, 8:54am UTC](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665/17 "2016-08-26T08:54:24Z")

</div>

```nohighlight
if [host] in ["example1.sit.com", "example2.sit.com"] {
  elasticsearch {
    ...
    index => "beats_sit-%{+YYYY.MM.dd}"
  }
} else {
  elasticsearch {
    ...
    index => "beats_dev-%{+YYYY.MM.dd}"
  }
}

```

---

<div class="post-metadata">

**Author:** ![rocky4bmw](https://avatars.discourse-cdn.com/v4/letter/r/6bbea6/32.png) [@rocky4bmw](https://discuss.elastic.co/u/rocky4bmw)\
**Post date:** [August 26, 2016, 11:41am UTC](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665/18 "2016-08-26T11:41:16Z")

</div>

@magnusbaeck you are brilliant and now multiple indexes is working. Thanks a million for your timely support. This is the bestever support I recieved with Elasticsearch team!!

One last thing if we have different types of beats like filebeat,topbeat and packetbeat how can we do this. Now all beats are moving to one index as per environment.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 26, 2016, 1:56pm UTC](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665/19 "2016-08-26T13:56:24Z")

</div>

But that's what you did previously:

```
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
```

---

<div class="post-metadata">

**Author:** ![Vijayant\_Panda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vijayant_panda/32/18963_2.png) [@Vijayant\_Panda](https://discuss.elastic.co/u/Vijayant_Panda)\
**Post date:** [June 12, 2017, 7:29am UTC](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665/20 "2017-06-12T07:29:55Z")

</div>

Hi can you please share your logstash conf,filebeats conf file ?? How is the index -beats\_sit-YYYYMMDD working in kibana .have u created a template named beats\_sit\*.

[Next page](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665.md?page=2)