Filebeat netflow module do not start after update to 7.11

Elastic Stack Beats
#1

Hi,
after updating the stack from 7.10.2 to 7.11 the netflow module will not start anymore.
The only message we see every 10 seconds in journal is

filebeat[1976620]: 2021-02-11T17:40:01.242+0100        ERROR        [reload]        cfgfile/list.go:99        Error creating runner from config: Error getting config for fileset netflow/log: Error interpreting the template of the input: template: text:8:5: executing "text" at <.internal_networks>: map has no entry for key "internal_networks"

Hoppe that sombody can help.

Best regards
Uwe

#2

was able to fix it.
With 7.11 the ../modules/netflow.yml changed

I had to add

      internal_networks:
        - private

in my existing yml file

#3

Just had the same issue.
The new variable is neither documented nor a note in the upgrade guide or breaking changes!

#4

I opened a bug for the startup failure and missing docs for the new variable: [filebeat][netflow] filebeat does't start when internal_networks variable isn't defined in config file · Issue #24094 · elastic/beats · GitHub

#5

I'm running filebeat 7.11.2 in docker/docker-compose and am getting the same error still.

Error:

2021-03-12T03:38:31.569Z INFO [publisher] pipeline/module.go:113 Beat name: b0947ece6aad
2021-03-12T03:38:31.571Z INFO beater/filebeat.go:117 Enabled modules/filesets: netflow (log), ()
2021-03-12T03:38:31.571Z INFO instance/beat.go:437 filebeat stopped.
2021-03-12T03:38:31.571Z ERROR instance/beat.go:971 Exiting: Error getting config for fileset netflow/log: Error interpreting the template of the input: template: text:8:5: executing "text" at <.internal_networks>: map has no entry for key "internal_networks"

Below are my configs:

Docker-compose:

filebeat:
image: docker.elastic.co/beats/filebeat:7.11.2
command: filebeat -e -strict.perms=false
volumes:
  - type: bind
    source: ./beats/filebeat.yml
    target: /usr/share/filebeat/filebeat.yml
    read_only: true
  - type: bind
    source: ./beats/netflow.yml
    target: /usr/share/filebeat/netflow.yml
    read_only: true
ports:
  - "2055:2055"
networks:
  - elk
depends_on:
  - elasticsearch

filebeat.yml:

filebeat.config.modules:
  enabled: true
  path: ${path.config}/modules.d/*.yml

filebeat.modules:
  - module: netflow

setup.template.enabled: true

output.elasticsearch:
  hosts: ["whatever:9200"]

netflow.yml:

- module: netflow
  log:
    enabled: true
    var:
      netflow_host: 0.0.0.0
      netflow_port: 2055
      # internal_networks specifies which networks are considered internal or private
      # you can specify either a CIDR block or any of the special named ranges listed
      # at: https://www.elastic.co/guide/en/beats/filebeat/current/defining-processors.html#condition-network
      internal_networks:
        - private
#6

Dear All,
I have a similar issue. I upgraded from 7.10.2 to 7.11.2
Netflow didn't work, but found this article and changed /etc/filebeat/modules.d/netflow.yml
This is the content:

# Module: netflow
# Docs: https://www.elastic.co/guide/en/beats/filebeat/7.10/filebeat-module-netflow.html

- module: netflow
  log:
    enabled: true
    var:
      netflow_host: 0.0.0.0
      netflow_port: 2055
      internal_networks:
        - private

When I run filebeat setup --modules netflow I get this error

Exiting: Error getting config for fileset netflow/log: Error interpreting the template of the input: template: text:8:5: executing "text" at <.internal_networks>: map has no entry for key "internal_networks"

Any help is welcome.

Kind regards
Hans

#7

@mayer , I am also on 7.11.2 but it is working. The only diffrence is that I use the interface adresse instead of 0.0.0.0

Best regards
Uwe

#8

Hi Uwe,

thanks coming back to my question. I tried your suggestion but it didn't help. But I have a workaround. I am not sure if good or bad, but it works. I restored the file /usr/share/filebeat/module/netflow/log/config/netflow.yml from my backup before the time of upgrade. And now it's working. This is the difference which was inserted in the last release:

7a8,14
> {{if .internal_networks}}
> internal_hosts:
> {{range .internal_networks}}
> - '{{ . }}'
> {{end}}
> {{end}}
>

// Hans

#9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.