Hi there,
i created my own filebeat module, "filebeat-modules-devguide" served as the basis.
All tests had been successful and now wanted to test them in real.
The index and the ingest pipelines are created successfully, also a UDP server is started on the corresponding filebeat port. However, no data is received from the UDP port, even though it is sent and also arrives (tested with tcpdump).
2020-06-02T17:51:13.013+0200 INFO cfgfile/reload.go:164 Config reloader started
2020-06-02T17:51:13.013+0200 DEBUG [cfgfile] cfgfile/reload.go:194 Scan for new config files
2020-06-02T17:51:13.013+0200 DEBUG [cfgfile] cfgfile/cfgfile.go:193 Load config from file: /etc/filebeat/modules.d/sophosxg.yml
2020-06-02T17:51:13.013+0200 DEBUG [cfgfile] cfgfile/reload.go:213 Number of module configs found: 1
2020-06-02T17:51:13.013+0200 DEBUG [reload] cfgfile/list.go:62 Starting reload procedure, current runners: 0
2020-06-02T17:51:13.013+0200 DEBUG [reload] cfgfile/list.go:80 Start list: 1, Stop list: 0
2020-06-02T17:51:13.014+0200 DEBUG [processors] processors/processor.go:101 Generated new processors: add_locale=[format=offset]
2020-06-02T17:51:13.014+0200 DEBUG [reload] cfgfile/list.go:101 Starting runner: sophosxg (firewall)
2020-06-02T17:51:13.014+0200 INFO eslegclient/connection.go:97 elasticsearch url: https://172.16.34.221:9200
2020-06-02T17:51:13.014+0200 WARN [tls] tlscommon/tls_config.go:83 SSL/TLS verifications disabled.
2020-06-02T17:51:13.014+0200 DEBUG [esclientleg] eslegclient/connection.go:284 ES Ping(url=https://172.16.34.221:9200)
2020-06-02T17:51:13.015+0200 WARN [tls] tlscommon/tls_config.go:83 SSL/TLS verifications disabled.
2020-06-02T17:51:13.036+0200 DEBUG [esclientleg] eslegclient/connection.go:307 Ping status code: 200
2020-06-02T17:51:13.036+0200 INFO [esclientleg] eslegclient/connection.go:308 Attempting to connect to Elasticsearch version 7.7.0
2020-06-02T17:51:13.036+0200 DEBUG [modules] fileset/pipelines.go:67 Required processors: [{geoip ingest-geoip}]
2020-06-02T17:51:13.036+0200 DEBUG [esclientleg] eslegclient/connection.go:358 GET https://172.16.34.221:9200/_nodes/ingest <nil>
2020-06-02T17:51:13.051+0200 DEBUG [esclientleg] eslegclient/connection.go:358 GET https://172.16.34.221:9200/_ingest/pipeline/filebeat-8.0.0-sophosxg-firewall-pipeline <nil>
2020-06-02T17:51:13.056+0200 DEBUG [modules] fileset/pipelines.go:120 Pipeline filebeat-8.0.0-sophosxg-firewall-pipeline already loaded
2020-06-02T17:51:13.056+0200 DEBUG [esclientleg] eslegclient/connection.go:358 GET https://172.16.34.221:9200/_ingest/pipeline/filebeat-8.0.0-sophosxg-firewall-firewall <nil>
2020-06-02T17:51:13.062+0200 DEBUG [modules] fileset/pipelines.go:120 Pipeline filebeat-8.0.0-sophosxg-firewall-firewall already loaded
2020-06-02T17:51:13.062+0200 DEBUG [esclientleg] eslegclient/connection.go:358 GET https://172.16.34.221:9200/_ingest/pipeline/filebeat-8.0.0-sophosxg-firewall-idp <nil>
2020-06-02T17:51:13.065+0200 DEBUG [modules] fileset/pipelines.go:120 Pipeline filebeat-8.0.0-sophosxg-firewall-idp already loaded
2020-06-02T17:51:13.065+0200 DEBUG [esclientleg] eslegclient/connection.go:358 GET https://172.16.34.221:9200/_ingest/pipeline/filebeat-8.0.0-sophosxg-firewall-atp <nil>
2020-06-02T17:51:13.069+0200 DEBUG [modules] fileset/pipelines.go:120 Pipeline filebeat-8.0.0-sophosxg-firewall-atp already loaded
2020-06-02T17:51:13.069+0200 DEBUG [esclientleg] eslegclient/connection.go:358 GET https://172.16.34.221:9200/_ingest/pipeline/filebeat-8.0.0-sophosxg-firewall-antivirus <nil>
2020-06-02T17:51:13.073+0200 DEBUG [modules] fileset/pipelines.go:120 Pipeline filebeat-8.0.0-sophosxg-firewall-antivirus already loaded
2020-06-02T17:51:13.073+0200 DEBUG [esclientleg] eslegclient/connection.go:358 GET https://172.16.34.221:9200/_ingest/pipeline/filebeat-8.0.0-sophosxg-firewall-sandstorm <nil>
2020-06-02T17:51:13.075+0200 DEBUG [modules] fileset/pipelines.go:120 Pipeline filebeat-8.0.0-sophosxg-firewall-sandstorm already loaded
2020-06-02T17:51:13.075+0200 DEBUG [esclientleg] eslegclient/connection.go:358 GET https://172.16.34.221:9200/_ingest/pipeline/filebeat-8.0.0-sophosxg-firewall-cfilter <nil>
2020-06-02T17:51:13.079+0200 DEBUG [modules] fileset/pipelines.go:120 Pipeline filebeat-8.0.0-sophosxg-firewall-cfilter already loaded
2020-06-02T17:51:13.079+0200 DEBUG [esclientleg] eslegclient/connection.go:358 GET https://172.16.34.221:9200/_ingest/pipeline/filebeat-8.0.0-sophosxg-firewall-event <nil>
2020-06-02T17:51:13.082+0200 DEBUG [modules] fileset/pipelines.go:120 Pipeline filebeat-8.0.0-sophosxg-firewall-event already loaded
2020-06-02T17:51:13.082+0200 INFO cfgfile/reload.go:224 Loading of config files completed.
2020-06-02T17:51:13.082+0200 INFO udp/input.go:103 Starting UDP input
manifest:
module_version: 1.0
var:
- name: syslog_host
default: localhost
- name: tags
default: [sophosxg-firewall]
- name: syslog_port
default: 9005
- name: input
default: udp
ingest_pipeline:
- ingest/pipeline.yml
- ingest/firewall.yml
- ingest/idp.yml
- ingest/atp.yml
- ingest/antivirus.yml
- ingest/sandstorm.yml
- ingest/cfilter.yml
- ingest/event.yml
input: config/firewall.yml
requires.processors:
- name: geoip
plugin: ingest-geoip
config input.yml:
{{ if eq .input "tcp" }}
type: tcp
host: "{{.syslog_host}}:{{.syslog_port}}"
{{ else if eq .input "udp" }}
type: udp
host: "{{.syslog_host}}:{{.syslog_port}}"
{{ else if eq .input "file" }}
type: log
paths:
{{ range $i, $path := .paths }}
- {{$path}}
{{ end }}
exclude_files: [".gz$"]
{{ end }}
tags: {{.tags}}
processors:
- add_locale: ~
What am i doing wrong here ?
Thanks for any help.
StefanS