Using processor in Filebeat Nginx module

Elastic Stack Beats
,
1

I am trying to drop some fields on the indices ingested by the filebeat Nginx module.
I followed the instructions on the thread How to use processors in filebeat HAproxy's module? - #3 by aventrax, but the fields are not being dropped when I restart filebeat with "filebeat -e".

My config looks like this:

- module: nginx
  # Access logs
  access:
    enabled: true

# Set custom paths for the log files. If left empty,
# Filebeat will choose the paths depending on your OS.
#var.paths:

# Input configuration (advanced). Any input configuration option
# can be added under this section.
input:
  processors:
    - drop_fields:
        fields: ["agent", "user_agent"]

Am I missing anything? Any help is appreciated!

Thanks,
Jenna

1 Like
2

The indentation seems incorrect. Try this:

- module: nginx
  # Access logs
  access:
    enabled: true

  input:
    processors:
      - drop_fields:
          fields: ["agent", "user_agent"]
3

Hi Noémi,

Thank you for your response! I tried it and it threw an error: > "Exiting: Failed to start crawler: creating module reloader failed: fileset nginx/input is configured but doesn't exist

Exiting: Failed to start crawler: creating module reloader failed: fileset nginx/input is configured but doesn't exist"

I am trying to configure the input for access only which is why the input is indented under the access block, the error log and the ingress_controller are not enabled. The full config looks like this:
# Module: nginx
# Docs: Nginx module | Filebeat Reference [7.10] | Elastic

- module: nginx
  # Access logs
  access:
    enabled: true

    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    #var.paths:

    # Input configuration (advanced). Any input configuration option
    # can be added under this section.
    input:
      processors:
        - drop_fields:
            fields: ["agent", "user_agent"]


  # Error logs
  error:
    enabled: false

    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    #var.paths:

  # Ingress-nginx controller logs. This is disabled by default. It could be used in Kubernetes environments to parse ingress-nginx logs
  ingress_controller:
    enabled: false

    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    #var.paths:

Best,
Jenna

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.