# How to configure conditional input-output from Filebeats

**URL:** <https://discuss.elastic.co/t/how-to-configure-conditional-input-output-from-filebeats/143121>\
**Category:** Logstash\
**Created:** [August 6, 2018, 8:29am UTC](https://discuss.elastic.co/t/how-to-configure-conditional-input-output-from-filebeats/143121 "2018-08-06T08:29:51Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![74db36a597f21b891b3f](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/74db36a597f21b891b3f/32/45367_2.png) [@74db36a597f21b891b3f](https://discuss.elastic.co/u/74db36a597f21b891b3f)\
**Post date:** [August 6, 2018, 8:29am UTC](https://discuss.elastic.co/t/how-to-configure-conditional-input-output-from-filebeats/143121/1 "2018-08-06T08:29:51Z")

</div>

Hi there. I'm a newbie in ELK stack. I am trying to configure logstasth to gather data from filebeat and put it in different indices depending from sources' filenames.

Filebeats config:

```
filebeat.inputs:
- type: log
enabled: true
paths:
   - D:\Logs\UIS\CMS\*
fields:
log_type: cmslog
fields_under_root: true
- type: log
enabled: true
paths:
   - D:\Logs\UIS\MonitoringService\*
fields:
log_type: monlog
fields_under_root: true

```

Logstash config:

```
input {
  beats {
    port => 5044
  }
}
filter{
 if [fields.log_type] == "cmslog" {
  grok{
  match=>{ "message" => "%{DATE_EU:date}\s*%{TIME:time}\s*\[%{DATA:thread}\]\s*\[%{DATA:username}\]\s*\[%{LOGLEVEL:loglevel}\]\s*\[%{DATA:logger}\]\s*\[%{DATA:someguid}\]\s*%{NO$
  }
 }
}
output {
  if [fields.log_type=="cmslog"]{
  elasticsearch {
    hosts => ["host:9200"]
    sniffing => true
    index => "cmslogs-%{+YYYY.MM.dd}"
    manage_template => false
   }
  }
}

```

With these configs Elasticsearch get no data.  
How to write conditions correct or debug why it's not working?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 6, 2018, 8:44am UTC](https://discuss.elastic.co/t/how-to-configure-conditional-input-output-from-filebeats/143121/2 "2018-08-06T08:44:21Z")

</div>

You're using the wrong syntax for nested fields, see [https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html#logstash-config-field-references](https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html#logstash-config-field-references).

---

<div class="post-metadata">

**Author:** ![74db36a597f21b891b3f](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/74db36a597f21b891b3f/32/45367_2.png) [@74db36a597f21b891b3f](https://discuss.elastic.co/u/74db36a597f21b891b3f)\
**Post date:** [August 6, 2018, 8:58am UTC](https://discuss.elastic.co/t/how-to-configure-conditional-input-output-from-filebeats/143121/3 "2018-08-06T08:58:48Z")

</div>

Hello, **magnusbaeck**.

Thank for the reference. Should I use [log\_type=="cmslog"] instead of [fields.log\_type=="cmslog"].  
This config is not working too. Is my custom field on top-level. And if not, which field is on top?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 6, 2018, 9:37am UTC](https://discuss.elastic.co/t/how-to-configure-conditional-input-output-from-filebeats/143121/4 "2018-08-06T09:37:14Z")

</div>

Since you have `fields_under_root: true` you should use `[log_type] == "cmslog"`. But there's no need to speculate; skip the conditionals and inspect what your events actually look like, then adjust your configuration to suit reality.

---

<div class="post-metadata">

**Author:** ![74db36a597f21b891b3f](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/74db36a597f21b891b3f/32/45367_2.png) [@74db36a597f21b891b3f](https://discuss.elastic.co/u/74db36a597f21b891b3f)\
**Post date:** [August 6, 2018, 2:12pm UTC](https://discuss.elastic.co/t/how-to-configure-conditional-input-output-from-filebeats/143121/5 "2018-08-06T14:12:22Z")

</div>

Without filters my data just comes through logstash without problem.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 6, 2018, 2:24pm UTC](https://discuss.elastic.co/t/how-to-configure-conditional-input-output-from-filebeats/143121/6 "2018-08-06T14:24:42Z")

</div>

Yes, but what does an example event look like?

---

<div class="post-metadata">

**Author:** ![74db36a597f21b891b3f](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/74db36a597f21b891b3f/32/45367_2.png) [@74db36a597f21b891b3f](https://discuss.elastic.co/u/74db36a597f21b891b3f)\
**Post date:** [August 6, 2018, 2:34pm UTC](https://discuss.elastic.co/t/how-to-configure-conditional-input-output-from-filebeats/143121/7 "2018-08-06T14:34:41Z")

</div>

All data is being dropped to one index. Even grok filter is working.  
Problem is to conditionally allocate different log by indices.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 6, 2018, 5:48pm UTC](https://discuss.elastic.co/t/how-to-configure-conditional-input-output-from-filebeats/143121/9 "2018-08-06T17:48:20Z")

</div>

Yes, I understand what the problem is. If you want help to resolve this please answer my questions.

---

<div class="post-metadata">

**Author:** ![74db36a597f21b891b3f](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/74db36a597f21b891b3f/32/45367_2.png) [@74db36a597f21b891b3f](https://discuss.elastic.co/u/74db36a597f21b891b3f)\
**Post date:** [August 7, 2018, 8:19am UTC](https://discuss.elastic.co/t/how-to-configure-conditional-input-output-from-filebeats/143121/10 "2018-08-07T08:19:49Z")

</div>

Hello. Sorry for misunderstanding.  
Here is the sample event.

 ![pic2](https://us1.discourse-cdn.com/elastic/original/3X/6/7/67a7b098bb618210f91a96eca0a781a5891e259d.PNG)

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 7, 2018, 8:47am UTC](https://discuss.elastic.co/t/how-to-configure-conditional-input-output-from-filebeats/143121/11 "2018-08-07T08:47:57Z")

</div>

You have misindented your `field_under_root: true` line in the Filebeat configuration. It should be on the same level as the `fields:` lines. If you fix that `fields.log_type` will become plain `log_type` and your Logstash configuration should read `if [log_type] = "..." {`.

---

<div class="post-metadata">

**Author:** ![74db36a597f21b891b3f](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/74db36a597f21b891b3f/32/45367_2.png) [@74db36a597f21b891b3f](https://discuss.elastic.co/u/74db36a597f21b891b3f)\
**Post date:** [August 7, 2018, 12:48pm UTC](https://discuss.elastic.co/t/how-to-configure-conditional-input-output-from-filebeats/143121/12 "2018-08-07T12:48:42Z")

</div>

Thank you very much. It helps.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 4, 2018, 12:48pm UTC](https://discuss.elastic.co/t/how-to-configure-conditional-input-output-from-filebeats/143121/13 "2018-09-04T12:48:44Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.