# Using processors with modules in Filebeat

**URL:** <https://discuss.elastic.co/t/using-processors-with-modules-in-filebeat/251001>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [October 5, 2020, 1:56pm UTC](https://discuss.elastic.co/t/using-processors-with-modules-in-filebeat/251001 "2020-10-05T13:56:10Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![DougR](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dougr/32/48095_2.png) [@DougR](https://discuss.elastic.co/u/DougR)\
**Post date:** [October 5, 2020, 1:56pm UTC](https://discuss.elastic.co/t/using-processors-with-modules-in-filebeat/251001/1 "2020-10-05T13:56:10Z")

</div>

# TL;DR

How do I add fields (or any processors) to the config for a preexisting module without editing the module source?

# Issue

I'm attempting to add some fields to logs ingested via the `system` module. This is my `modules.d/system.yml`:

```auto
# Module: system
# Docs: https://www.elastic.co/guide/en/beats/filebeat/7.9/filebeat-module-system.html

- module: system
  # Syslog
  syslog:
    enabled: true

    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    #var.paths:

    # Add additional required fields.
    processors:
      - add_fields:
          target: ''
          fields:
            application:
              name: 'myapp'
              env: 'dev'

  # Authorization logs
  auth:
    enabled: true

    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    #var.paths:

    # Add additional required fields.
    processors:
      - add_fields:
          target: ''
          fields:
            application:
              name: 'myapp'
              env: 'dev'

```

This loads correctly, but the additional fields I add aren't available. When I look at (for example) `module/system/syslog/config/syslog.yml`, I see:

```auto
type: log
paths:
{{ range $i, $path := .paths }}
 - {{$path}}
{{ end }}
exclude_files: [".gz$"]
multiline:
  pattern: "^\\s"
  match: after
processors:
  - add_locale: ~
  - add_fields:
      target: ''
      fields:
        ecs.version: 1.5.0

```

which makes me question whether this is possible, without editing that file, which isn't desirable, since it gets overwritten each time I update the filebeat, whereas `modules.d/system.yml` does not.

So my questions are...is it possible to add fields (or any processors) to the config for a preexisting module without editing the module source, and how?

---

<div class="post-metadata">

**Author:** ![Frank\_Barton](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/frank_barton/32/76687_2.png) [@Frank\_Barton](https://discuss.elastic.co/u/Frank_Barton)\
**Post date:** [October 5, 2020, 7:54pm UTC](https://discuss.elastic.co/t/using-processors-with-modules-in-filebeat/251001/2 "2020-10-05T19:54:13Z")

</div>

I am having a very similar problem - Specifically that the fingerprint processor doesn't seem to be running.

Here's my modules.d/netflow.yml

```auto
# Module: netflow
# Docs: https://www.elastic.co/guide/en/beats/filebeat/7.9/filebeat-module-netflow.html

- module: netflow
  log:
    enabled: true
    var:
      netflow_host: 0.0.0.0
      netflow_port: 2055
      processors:
        - fingerprint:
            fields: ["netflow.destination_ipv4_address","netflow.destination_transport_port","netflow.exporter.timestamp","netflow.source_ipv4_address","netflow.source_transport_port"]
            target_field: "@metadata._id"

```

I've done a little digging, and it seems that there is something of a disconnect as to how the modules parses/processes the inputs vs. how the type:netflow (in my case) parses

---

<div class="post-metadata">

**Author:** ![Frank\_Barton](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/frank_barton/32/76687_2.png) [@Frank\_Barton](https://discuss.elastic.co/u/Frank_Barton)\
**Post date:** [October 5, 2020, 8:41pm UTC](https://discuss.elastic.co/t/using-processors-with-modules-in-filebeat/251001/3 "2020-10-05T20:41:28Z")

</div>

Try this - I had to put an input stanza on which the processor worked

```auto
# Module: netflow
# Docs: https://www.elastic.co/guide/en/beats/filebeat/7.9/filebeat-module-netflow.html

- module: netflow
  log:
    enabled: true
    var:
      netflow_host: 0.0.0.0
      netflow_port: 2055
    input:
      processors:
        - fingerprint:
            fields: ["netflow.destination_ipv4_address","netflow.destination_transport_port","netflow.exporter.timestamp","netflow.source_ipv4_address","netflow.source_transport_port"]
            target_field: "@metadata._id"

```

---

<div class="post-metadata">

**Author:** ![DougR](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dougr/32/48095_2.png) [@DougR](https://discuss.elastic.co/u/DougR)\
**Post date:** [October 6, 2020, 1:03pm UTC](https://discuss.elastic.co/t/using-processors-with-modules-in-filebeat/251001/4 "2020-10-06T13:03:55Z")

</div>

> [@Frank\_Barton](#):
>
> Try this - I had to put an input stanza on which the processor worked
> 
> ```auto
> # Module: netflow
> # Docs: https://www.elastic.co/guide/en/beats/filebeat/7.9/filebeat-module-netflow.html
> 
> - module: netflow
> log:
> enabled: true
> var:
> netflow_host: 0.0.0.0
> netflow_port: 2055
> input:
> processors:
> - fingerprint:
> fields: ["netflow.destination_ipv4_address","netflow.destination_transport_port","netflow.exporter.timestamp","netflow.source_ipv4_address","netflow.source_transport_port"]
> target_field: "@metadata._id"
> 
> ```

I'll try this - thanks. I've noticed that modules already have some processors specified (at least the `system` module, haven't looked at `netflow`). Have you noticed - does doing this override those processors, or does that data still appear?

---

<div class="post-metadata">

**Author:** ![Frank\_Barton](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/frank_barton/32/76687_2.png) [@Frank\_Barton](https://discuss.elastic.co/u/Frank_Barton)\
**Post date:** [October 6, 2020, 2:27pm UTC](https://discuss.elastic.co/t/using-processors-with-modules-in-filebeat/251001/5 "2020-10-06T14:27:09Z")

</div>

I haven't looked at the system module much yet - I will do so shortly.

I'll be honest, I just started my elastic journey last week, and have been beating my head against some of the documentation. I hope that my struggles can help some other folks.

Frank

---

<div class="post-metadata">

**Author:** ![DougR](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dougr/32/48095_2.png) [@DougR](https://discuss.elastic.co/u/DougR)\
**Post date:** [October 6, 2020, 6:15pm UTC](https://discuss.elastic.co/t/using-processors-with-modules-in-filebeat/251001/6 "2020-10-06T18:15:19Z")

</div>

> [@Frank\_Barton](#):
>
> I haven't looked at the system module much yet - I will do so shortly.
> 
> I'll be honest, I just started my elastic journey last week, and have been beating my head against some of the documentation. I hope that my struggles can help some other folks.
> 
> Frank

I'll be looking a the `system` module here shortly, will keep you updated. Welcome to the Elastic fellowship!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 3, 2020, 8:15pm UTC](https://discuss.elastic.co/t/using-processors-with-modules-in-filebeat/251001/7 "2020-11-03T20:15:39Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.