# Filebeat processors not working as expected

**URL:** <https://discuss.elastic.co/t/filebeat-processors-not-working-as-expected/135022>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [June 7, 2018, 3:07pm UTC](https://discuss.elastic.co/t/filebeat-processors-not-working-as-expected/135022 "2018-06-07T15:07:50Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Vikrant\_Aggarwal](https://avatars.discourse-cdn.com/v4/letter/v/f14d63/32.png) [@Vikrant\_Aggarwal](https://discuss.elastic.co/u/Vikrant_Aggarwal)\
**Post date:** [June 7, 2018, 3:07pm UTC](https://discuss.elastic.co/t/filebeat-processors-not-working-as-expected/135022/1 "2018-06-07T15:07:51Z")

</div>

Here is my filebeat.yml file.

#### First Issue : It was working fine with single processor when I was not testing the and condition, as soon as I added the and condition. I am getting error.

```auto
$ ./filebeat
Exiting: error loading config file: yaml: line 31: found character that cannot start any token

```

```auto
$ egrep -v "^(#| #|$)" filebeat.yml
filebeat.prospectors:
- type: log
  enabled: true
  paths:
    - /Users/viaggarw/Documents/ELK/exercise_1.log
  processors:
    - drop_event:
        when: <<<<<< Line 31
	  and:
            - contains:
                message: "INFO"
            - contains:
                message: "start"
    #- drop_fields:
    # fields: ["offset"]
    # when:
    #- c:\programdata\elasticsearch\logs\*
  exclude_files: ['.gz$']
filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false
setup.template.settings:
  index.number_of_shards: 3
setup.kibana:
output.console:
  pretty: true

```

my log file is 🙂

```auto
$ cat exercise_1.log
[DEBUG] 2018-12-07 03:57:27.064 [https-jsse-nio-8443-exec-10] RequestProcessor - Status >200
[INFO] 2018-12-07 04:00:41.015 [main] Application - Starting Application v0.0.1-SNAPSHOT on c62e0ddde6f5 with PID 141 (/etc/app1/bin/app1.jar started by app1 in /etc/app1/bin)
[DEBUG] 2018-12-07 04:00:41.022 [main] Application - Running with Spring Boot v1.5.2.RELEASE, Spring v4.3.7.RELEASE
[INFO] 2018-12-07 04:00:41.023 [main] Application - No active profile set, falling back to default profiles: default
[INFO] 2018-12-07 04:00:44.265 [main] Application - Started Application in 3.824 seconds (JVM running for 4.755)
[DEBUG] 2018-12-07 18:31:09.868 [https-jsse-nio-8443-exec-4] RestProcessor - https://127.0.0.1:8200/v1/auth/ldap/login/user1
[DEBUG] 2018-12-07 18:31:10.241 [https-jsse-nio-8443-exec-4] RequestProcessor - Status >200

```

I know we may follow the other ways to get the end results but I am just trying to understand the working of processors.

#### Second Issue : When using the following filebeat.yml file.

```auto
$ egrep -v "^(#| #|$)" filebeat.yml | grep -v "#"
filebeat.prospectors:
- type: log
  enabled: true
  paths:
    - /Users/viaggarw/Documents/ELK/exercise_1.log
  processors:
    - drop_event:
        when:
          contains:
            message: "INFO"
    - drop_fields:
        fields: ["offset"]
        when: <<<<< Line no 36
	  contains:
	    message: "DEBUG"
  exclude_files: ['.gz$']
filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false
setup.template.settings:
  index.number_of_shards: 3
setup.kibana:
output.console:
  pretty: true

```

Found this issue:

```auto
$ ./filebeat
Exiting: error loading config file: yaml: line 36: found character that cannot start any token

```

I am not sure what I am doing wrong. Can anyone please help to identify the issue?

---

<div class="post-metadata">

**Author:** ![adrisr](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adrisr/32/25423_2.png) [@adrisr](https://discuss.elastic.co/u/adrisr)\
**Post date:** [June 7, 2018, 3:57pm UTC](https://discuss.elastic.co/t/filebeat-processors-not-working-as-expected/135022/2 "2018-06-07T15:57:42Z")

</div>

Both issues are caused by bad indentation. You are mixing spaces and tabs. It might look well indented in your editor but as you can see when pasted here it looks wrong.

It's [recommended](https://www.elastic.co/guide/en/beats/winlogbeat/current/yaml-tips.html) to use only use spaces with yaml.

---

<div class="post-metadata">

**Author:** ![Vikrant\_Aggarwal](https://avatars.discourse-cdn.com/v4/letter/v/f14d63/32.png) [@Vikrant\_Aggarwal](https://discuss.elastic.co/u/Vikrant_Aggarwal)\
**Post date:** [June 7, 2018, 4:30pm UTC](https://discuss.elastic.co/t/filebeat-processors-not-working-as-expected/135022/3 "2018-06-07T16:30:41Z")

</div>

Thanks. It helps to make progress.

```auto
$ egrep -v "^(#| #|$)" filebeat.yml | grep -v "#"
filebeat.prospectors:
- type: log
  enabled: true
  paths:
    - /Users/viaggarw/Documents/ELK/exercise_1.log
  processors:
    - drop_event:
      when:
        contains:
          message: "INFO"
    - drop_fields:
      fields: ["offset"]
      when:
        contains:
          message: "DEBUG"
  exclude_files: ['.gz$']
filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false
setup.template.settings:
  index.number_of_shards: 3
setup.kibana:
output.console:
  pretty: true

```

Verified that yaml is having proper indentation.

```auto
$ python -c 'import yaml,sys;yaml.safe_load(sys.stdin)' < filebeat.yml
$

```

It's giving me this error now.

```auto
$ ./filebeat
Exiting: Error in initing prospector: each processor needs to have exactly one action, but found 2 actions

```

As per this [link](https://www.elastic.co/guide/en/beats/filebeat/current/defining-processors.html) it should work.

---

<div class="post-metadata">

**Author:** ![adrisr](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adrisr/32/25423_2.png) [@adrisr](https://discuss.elastic.co/u/adrisr)\
**Post date:** [June 15, 2018, 4:24am UTC](https://discuss.elastic.co/t/filebeat-processors-not-working-as-expected/135022/4 "2018-06-15T04:24:57Z")

</div>

Hi,

Your config was still not OK according to the link you provided, the difference is subtle but important.

You need to add an extra level of indent to the contents of `- drop_event:` and `- drop_fields`, like this:

```auto
processors:
    - drop_event:
        when:
          contains:
            message: "INFO"
    - drop_fields:
        fields: ["offset"]
        when:
          contains:
            message: "DEBUG"

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 13, 2018, 4:38am UTC](https://discuss.elastic.co/t/filebeat-processors-not-working-as-expected/135022/5 "2018-07-13T04:38:38Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.