# Issue with index

**URL:** <https://discuss.elastic.co/t/issue-with-index/78028>\
**Category:** Logstash\
**Created:** [March 9, 2017, 5:21pm UTC](https://discuss.elastic.co/t/issue-with-index/78028 "2017-03-09T17:21:26Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![bob-bza](https://avatars.discourse-cdn.com/v4/letter/b/9f8e36/32.png) [@bob-bza](https://discuss.elastic.co/u/bob-bza)\
**Post date:** [March 9, 2017, 5:21pm UTC](https://discuss.elastic.co/t/issue-with-index/78028/1 "2017-03-09T17:21:26Z")

</div>

**filebeat 1:**

- input\_type: log  
paths:
  - /var/log/ambari.log  
document\_type: ambari

- input\_type: log  
paths:
  - /var/log/zookeeper.log  
document\_type: zookeeper

tags: ["masternode"]

**filebeat 2:**

- input\_type: log  
paths:
  - /var/log/hive.log  
document\_type: hive

- input\_type: log  
paths:
  - /var/log/zookeeper.log  
document\_type: zookeeper

tags: ["datanode"]

**Logstash**  
input {  
beats {  
port =\> 5044  
}  
}

output{  
if "masternode" in [tags] {  
if [type] == "ambari" {  
elasticsearch {  
hosts =\> ["endpoint"]  
action =\> "index"  
index =\> "ambarinodelog"  
}  
}  
if [type] == "zookeeper" {  
elasticsearch {  
hosts =\> ["endpoint"]  
action =\> "index"  
index =\> "zookeeperlog"  
}  
}  
}  
if "masternode" in [tags] {  
if [type] == "hive" {  
elasticsearch {  
hosts =\> ["endpoint"]  
action =\> "index"  
index =\> "hivelog"  
}  
}  
if [type] == "zookeeper" {  
elasticsearch {  
hosts =\> ["endpoint"]  
action =\> "index"  
index =\> "zookeeperlog"  
}  
}  
}  
}

**what i was trying to was**  
i have multiple filebeats with different tags.  
want to send all zookeeper logs to single index irrespective of tags does it works?  
If yes please guide me proper required changes in my configuration

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [March 9, 2017, 6:45pm UTC](https://discuss.elastic.co/t/issue-with-index/78028/2 "2017-03-09T18:45:36Z")

</div>

Just don't put the elasticsearch output that writes to the zookeeperlog index inside the conditionals.

---

<div class="post-metadata">

**Author:** ![bob-bza](https://avatars.discourse-cdn.com/v4/letter/b/9f8e36/32.png) [@bob-bza](https://discuss.elastic.co/u/bob-bza)\
**Post date:** [March 9, 2017, 7:03pm UTC](https://discuss.elastic.co/t/issue-with-index/78028/3 "2017-03-09T19:03:29Z")

</div>

> [@magnusbaeck](#):
>
> Just don't put the elasticsearch output that writes to the zookeeperlog index inside the conditionals.

output{  
if "masternode" in [tags] {  
if [type] == "ambari" {  
elasticsearch {  
hosts =\> ["endpoint"]  
action =\> "index"  
index =\> "ambarinodelog"  
}  
}  
}  
if "masternode" in [tags] {  
if [type] == "hive" {  
elasticsearch {  
hosts =\> ["endpoint"]  
action =\> "index"  
index =\> "hivelog"  
}  
}  
}  
if [type] == "zookeeper" {  
elasticsearch {  
hosts =\> ["endpoint"]  
action =\> "index"  
index =\> "zookeeperlog"  
}  
}  
}

are you suggesting for this approch?

---

<div class="post-metadata">

**Author:** ![jkuang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jkuang/32/72637_2.png) [@jkuang](https://discuss.elastic.co/u/jkuang)\
**Post date:** [March 9, 2017, 7:46pm UTC](https://discuss.elastic.co/t/issue-with-index/78028/4 "2017-03-09T19:46:17Z")

</div>

FYI. The if condition for your hive and zookeeper is incorrect.

The second if statement should be

```auto

if "datanode" in [tags]

and not

if "masternode" in [tags]

```

since your filebeat2 is tags: ["datanode"]

---

<div class="post-metadata">

**Author:** ![bob-bza](https://avatars.discourse-cdn.com/v4/letter/b/9f8e36/32.png) [@bob-bza](https://discuss.elastic.co/u/bob-bza)\
**Post date:** [March 9, 2017, 8:04pm UTC](https://discuss.elastic.co/t/issue-with-index/78028/5 "2017-03-09T20:04:28Z")

</div>

> [@jkuang](#):
>
> FYI. The if condition for your hive and zookeeper is incorrect.
> 
> The second if statement should be
> 
> if "datanode" in [tags]
> 
> and not
> 
> if "masternode" in [tags]
> 
> since your filebeat2 is tags: ["datanode"]

can you please show in detail, i dont have much idea what you are telling about

---

<div class="post-metadata">

**Author:** ![jkuang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jkuang/32/72637_2.png) [@jkuang](https://discuss.elastic.co/u/jkuang)\
**Post date:** [March 9, 2017, 8:20pm UTC](https://discuss.elastic.co/t/issue-with-index/78028/6 "2017-03-09T20:20:28Z")

</div>

Look for datanode below:

```auto
output{
if "masternode" in [tags] {
if [type] == "ambari" {
elasticsearch {
hosts => ["endpoint"]
action => "index"
index => "ambarinodelog"
}
}
if [type] == "zookeeper" {
elasticsearch {
hosts => ["endpoint"]
action => "index"
index => "zookeeperlog"
}
}
}
if "datanode" in [tags] {
if [type] == "hive" {
elasticsearch {
hosts => ["endpoint"]
action => "index"
index => "hivelog"
}
}
if [type] == "zookeeper" {
elasticsearch {
hosts => ["endpoint"]
action => "index"
index => "zookeeperlog"
}
}
}
}

```

---

<div class="post-metadata">

**Author:** ![bob-bza](https://avatars.discourse-cdn.com/v4/letter/b/9f8e36/32.png) [@bob-bza](https://discuss.elastic.co/u/bob-bza)\
**Post date:** [March 9, 2017, 9:45pm UTC](https://discuss.elastic.co/t/issue-with-index/78028/7 "2017-03-09T21:45:54Z")

</div>

yes got you. thats an example i posted . sorry my mistake . i forget to add datanode in tags.

can i know exact syntax for same index(zookeeper) for multiple tags(datanode, slavenodes,masters,edge etc etc)

i want all zookeeper logs logs to same index

> [@jkuang](#):
>
> output{  
> if "masternode" in [tags] {  
> if [type] == "ambari" {  
> elasticsearch {  
> hosts =\> ["endpoint"]  
> action =\> "index"  
> index =\> "ambarinodelog"  
> }  
> }  
> if [type] == "zookeeper" {  
> elasticsearch {  
> hosts =\> ["endpoint"]  
> action =\> "index"  
> index =\> "zookeeperlog"  
> }  
> }  
> }  
> if "datanode" in [tags] {  
> if [type] == "hive" {  
> elasticsearch {  
> hosts =\> ["endpoint"]  
> action =\> "index"  
> index =\> "hivelog"  
> }  
> }  
> if [type] == "zookeeper" {  
> elasticsearch {  
> hosts =\> ["endpoint"]  
> action =\> "index"  
> index =\> "zookeeperlog"  
> }  
> }  
> }  
> }

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [March 10, 2017, 7:51am UTC](https://discuss.elastic.co/t/issue-with-index/78028/8 "2017-03-10T07:51:29Z")

</div>

If you want all Zookeeper logs sent to the same place regardless of tag, don't put your zookeeperlogs outputs inside any `if "whatever" in [tags]` conditionals.

---

<div class="post-metadata">

**Author:** ![jkuang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jkuang/32/72637_2.png) [@jkuang](https://discuss.elastic.co/u/jkuang)\
**Post date:** [March 10, 2017, 2:57pm UTC](https://discuss.elastic.co/t/issue-with-index/78028/11 "2017-03-10T14:57:00Z")

</div>

Please ignore my previous post. Use the syntax below:

```auto
output
{
if "masternode" in [tags] {
if [type] == "ambari" {
elasticsearch {
hosts => ["endpoint"]
action => "index"
index => "ambarinodelog"
}
}
}
if "datanode" in [tags] {
if [type] == "hive" {
elasticsearch {
hosts => ["endpoint"]
action => "index"
index => "hivelog"
}
}

}
if [type] == "zookeeper" {
elasticsearch {
hosts => ["endpoint"]
action => "index"
index => "zookeeperlog"
}
}
}

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 7, 2017, 2:57pm UTC](https://discuss.elastic.co/t/issue-with-index/78028/12 "2017-04-07T14:57:07Z")

</div>

Closed on April 7, 2017, 2:57pm UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.