Filebeat is connected to Kafka, but it doesn't send Log

CemG · April 26, 2022, 2:58pm

Hello,

It is been 2 days that I have this issue.
I would like to send log from a computer to elastic by the following way :
Filebeat → Kafka → Logstash → Elasticsearch
But Filebeat do not send log to Kafka and I don't understand why, I have tried on the same server it worked, and when I try to send log from on server to another, Kafka do not receive anything.
So I don't know why on localhost it works, and a server with Filebeat do not send to a server with Kafka.
Here is the config that I have done

filebeat.inputs:
  - type: filestream
    enabled: true
    id: "apache-"
    paths:
      - /var/log/apache2/access.log

# ============================== Filebeat modules ==============================

filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false

# ======================= Elasticsearch template setting =======================

setup.template.settings:
  index.number_of_shards: 1

# ================================= Output Kafka ==============================

output.kafka:
  codec.format:
    string: '%{[@timestamp]}%{[message]}'
  hosts: ["kafka-server:9092"]
  topic: 'apache'
  partition.round_robin:
    reachable_only: false
  required_acks: 1
  compression: gzip
  max_message_bytes: 1000000

# ================================= Processors =================================
processors:
  - add_host_metadata:
      when.not.contains.tags: forwarded

Thank you for your help

Andrea_Spacca · April 27, 2022, 12:56am

hello @CemG , welcome to the Elastic community

Could you start filebeat with debug logs and share the content?

The fact that the same configuration works properly on localhost let me think there is some networking issue: is the kafka-server resolvable to an IP address from the machine where filebeat runs?

Is the port 9092 accepting external connection on the kafka-server and the machine where filebeat runs is allowing external connection to kafka-server resolved ip address/port 9092?

CemG · April 27, 2022, 7:08am

Hello,
Here is the debug log

{"log.level":"info","@timestamp":"2022-04-27T09:01:31.394+0200","log.origin":{"file.name":"instance/beat.go","file.line":669},"message":"Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:01:31.394+0200","log.origin":{"file.name":"instance/beat.go","file.line":677},"message":"Beat ID: c37eb5db-c6f2-492c-a8a3-63d8aa695170","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:01:31.395+0200","log.logger":"seccomp","log.origin":{"file.name":"seccomp/seccomp.go","file.line":124},"message":"Syscall filter successfully installed","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:01:31.395+0200","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1047},"message":"Beat info","service.name":"filebeat","system_info":{"beat":{"path":{"config":"/etc/filebeat","data":"/var/lib/filebeat","home":"/usr/share/filebeat","logs":"/var/log/filebeat"},"type":"filebeat","uuid":"c37eb5db-c6f2-492c-a8a3-63d8aa695170"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-04-27T09:01:31.395+0200","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1056},"message":"Build info","service.name":"filebeat","system_info":{"build":{"commit":"271435c21bfd4e2e621d87c04f4b815980626978","libbeat":"8.1.3","time":"2022-04-19T09:29:51.000Z","version":"8.1.3"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-04-27T09:01:31.395+0200","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1059},"message":"Go runtime info","service.name":"filebeat","system_info":{"go":{"os":"linux","arch":"amd64","max_procs":3,"version":"go1.17.8"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-04-27T09:01:31.396+0200","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1063},"message":"Host info","service.name":"filebeat","system_info":{"host":{"architecture":"x86_64","boot_time":"2022-04-26T15:55:12+02:00","containerized":false,"name":"scanner","ip":["127.0.0.1/8","::1/128","192.168.60.43/24","fe80::250:56ff:fea5:c4b5/64"],"kernel_version":"4.19.0-20-amd64","mac":["00:50:56:a5:c4:b5"],"os":{"type":"linux","family":"debian","platform":"debian","name":"Debian GNU/Linux","version":"10 (buster)","major":10,"minor":0,"patch":0,"codename":"buster"},"timezone":"CEST","timezone_offset_sec":7200,"id":"3d9b037af48a488884677d254db8ca99"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-04-27T09:01:31.396+0200","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1092},"message":"Process info","service.name":"filebeat","system_info":{"process":{"capabilities":{"inheritable":null,"permitted":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"effective":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"bounding":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"ambient":null},"cwd":"/etc/filebeat","exe":"/usr/share/filebeat/bin/filebeat","name":"filebeat","pid":2785,"ppid":1085,"seccomp":{"mode":"filter","no_new_privs":true},"start_time":"2022-04-27T09:01:30.410+0200"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-04-27T09:01:31.396+0200","log.origin":{"file.name":"instance/beat.go","file.line":323},"message":"Setup Beat: filebeat; Version: 8.1.3","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:01:31.397+0200","log.logger":"publisher","log.origin":{"file.name":"pipeline/consumer.go","file.line":98},"message":"start pipeline event consumer","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:01:31.397+0200","log.logger":"publisher","log.origin":{"file.name":"pipeline/module.go","file.line":113},"message":"Beat name: scanner","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:01:31.397+0200","log.logger":"publisher","log.origin":{"file.name":"pipeline/queue_reader.go","file.line":48},"message":"pipeline event consumer queue reader: start","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:01:31.398+0200","log.origin":{"file.name":"fileset/modules.go","file.line":103},"message":"Enabled modules/filesets:  ()","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2022-04-27T09:01:31.398+0200","log.origin":{"file.name":"beater/filebeat.go","file.line":168},"message":"Filebeat is unable to load the ingest pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the ingest pipelines or are using Logstash pipelines, you can ignore this warning.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:01:31.398+0200","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":142},"message":"Starting metrics logging every 30s","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:01:31.398+0200","log.origin":{"file.name":"instance/beat.go","file.line":489},"message":"filebeat start running.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:01:31.399+0200","log.origin":{"file.name":"memlog/store.go","file.line":119},"message":"Loading data file of '/var/lib/filebeat/registry/filebeat' succeeded. Active transaction id=0","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:01:31.399+0200","log.origin":{"file.name":"memlog/store.go","file.line":124},"message":"Finished loading transaction log file for '/var/lib/filebeat/registry/filebeat'. Active transaction id=12","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2022-04-27T09:01:31.400+0200","log.origin":{"file.name":"beater/filebeat.go","file.line":285},"message":"Filebeat is unable to load the ingest pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the ingest pipelines or are using Logstash pipelines, you can ignore this warning.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:01:31.400+0200","log.logger":"registrar","log.origin":{"file.name":"registrar/registrar.go","file.line":109},"message":"States Loaded from registrar: 0","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:01:31.400+0200","log.logger":"crawler","log.origin":{"file.name":"beater/crawler.go","file.line":71},"message":"Loading Inputs: 1","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:01:31.400+0200","log.logger":"crawler","log.origin":{"file.name":"beater/crawler.go","file.line":117},"message":"starting input, keys present on the config: [filebeat.inputs.0.enabled filebeat.inputs.0.id filebeat.inputs.0.paths.0 filebeat.inputs.0.type]","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:01:31.401+0200","log.logger":"crawler","log.origin":{"file.name":"beater/crawler.go","file.line":148},"message":"Starting input (ID: 7770775948050077450)","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:01:31.401+0200","log.logger":"crawler","log.origin":{"file.name":"beater/crawler.go","file.line":106},"message":"Loading and starting Inputs completed. Enabled inputs: 1","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:01:31.401+0200","log.logger":"input.filestream","log.origin":{"file.name":"compat/compat.go","file.line":111},"message":"Input filestream starting","service.name":"filebeat","id":"apache-","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:01:31.402+0200","log.origin":{"file.name":"cfgfile/reload.go","file.line":164},"message":"Config reloader started","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:01:31.402+0200","log.logger":"file_watcher","log.origin":{"file.name":"filestream/fswatch.go","file.line":138},"message":"Start next scan","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:01:31.402+0200","log.origin":{"file.name":"cfgfile/reload.go","file.line":224},"message":"Loading of config files completed.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:01:32.404+0200","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":139},"message":"Connecting to kafka(10.60.101.11:9092)","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:01:32.404+0200","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":147},"message":"Connection to kafka(10.60.101.11:9092) established","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:01:41.403+0200","log.logger":"file_watcher","log.origin":{"file.name":"filestream/fswatch.go","file.line":138},"message":"Start next scan","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:01:51.402+0200","log.logger":"file_watcher","log.origin":{"file.name":"filestream/fswatch.go","file.line":138},"message":"Start next scan","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:02:01.403+0200","log.logger":"file_watcher","log.origin":{"file.name":"filestream/fswatch.go","file.line":138},"message":"Start next scan","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:02:01.408+0200","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":184},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cgroup":{"memory":{"id":"session-c2.scope","mem":{"limit":{"bytes":9223372036854771712},"usage":{"bytes":631767040}}}},"cpu":{"system":{"ticks":50,"time":{"ms":61}},"total":{"ticks":230,"time":{"ms":248},"value":230},"user":{"ticks":180,"time":{"ms":187}}},"handles":{"limit":{"hard":1048576,"soft":1024},"open":12},"info":{"ephemeral_id":"d049bf8a-eb87-4810-9de6-4f9ceade755b","uptime":{"ms":30077},"version":"8.1.3"},"memstats":{"gc_next":21737728,"memory_alloc":12623152,"memory_sys":37569544,"memory_total":51501208,"rss":132845568},"runtime":{"goroutines":39}},"filebeat":{"events":{"active":6,"added":6},"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0},"reloads":1,"scans":1},"output":{"events":{"active":6,"batches":3,"failed":12,"total":18},"type":"kafka"},"outputs":{"kafka":{"bytes_read":1017,"bytes_write":288}},"pipeline":{"clients":1,"events":{"active":6,"published":6,"retry":18,"total":6},"queue":{"max_events":4096}}},"registrar":{"states":{"current":0}},"system":{"cpu":{"cores":3},"load":{"1":0.04,"15":0,"5":0.01,"norm":{"1":0.0133,"15":0,"5":0.0033}}}},"ecs.version":"1.6.0"}}
^C{"log.level":"info","@timestamp":"2022-04-27T09:02:05.535+0200","log.origin":{"file.name":"beater/filebeat.go","file.line":427},"message":"Stopping filebeat","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:02:05.535+0200","log.origin":{"file.name":"beater/crawler.go","file.line":155},"message":"Stopping Crawler","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:02:05.535+0200","log.origin":{"file.name":"beater/crawler.go","file.line":165},"message":"Stopping 1 inputs","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:02:05.535+0200","log.origin":{"file.name":"cfgfile/reload.go","file.line":227},"message":"Dynamic config reloader stopped","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:02:05.536+0200","log.logger":"crawler","log.origin":{"file.name":"beater/crawler.go","file.line":170},"message":"Stopping input: 7770775948050077450","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:02:05.536+0200","log.logger":"input.filestream","log.origin":{"file.name":"compat/compat.go","file.line":132},"message":"Input 'filestream' stopped","service.name":"filebeat","id":"apache-","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:02:05.536+0200","log.origin":{"file.name":"beater/crawler.go","file.line":185},"message":"Crawler stopped","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:02:05.536+0200","log.logger":"publisher","log.origin":{"file.name":"pipeline/client.go","file.line":158},"message":"client: closing acker","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:02:05.536+0200","log.logger":"input.filestream","log.origin":{"file.name":"filestream/input.go","file.line":320},"message":"Reader was closed. Closing.","service.name":"filebeat","id":"apache-","source_file":"filestream::apache-::native::523373-2054","path":"/var/log/apache2/access.log","state-id":"native::523373-2054","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:02:05.536+0200","log.logger":"publisher","log.origin":{"file.name":"pipeline/client.go","file.line":163},"message":"client: done closing acker","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:02:05.536+0200","log.logger":"publisher","log.origin":{"file.name":"pipeline/client.go","file.line":165},"message":"client: unlink from queue","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:02:05.536+0200","log.logger":"registrar","log.origin":{"file.name":"registrar/registrar.go","file.line":132},"message":"Stopping Registrar","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:02:05.536+0200","log.logger":"registrar","log.origin":{"file.name":"registrar/registrar.go","file.line":166},"message":"Ending Registrar","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:02:05.536+0200","log.logger":"registrar","log.origin":{"file.name":"registrar/registrar.go","file.line":137},"message":"Registrar stopped","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:02:05.536+0200","log.logger":"publisher","log.origin":{"file.name":"pipeline/client.go","file.line":187},"message":"client: cancelled 0 events","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:02:05.537+0200","log.logger":"publisher","log.origin":{"file.name":"pipeline/client.go","file.line":167},"message":"client: done unlink","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:02:05.537+0200","log.logger":"publisher","log.origin":{"file.name":"pipeline/client.go","file.line":170},"message":"client: closing processors","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:02:05.537+0200","log.logger":"publisher","log.origin":{"file.name":"pipeline/client.go","file.line":175},"message":"client: done closing processors","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:02:05.537+0200","log.logger":"input.filestream","log.origin":{"file.name":"compat/compat.go","file.line":124},"message":"Input 'filestream' stopped","service.name":"filebeat","id":"apache-","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:02:05.544+0200","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":192},"message":"Total metrics","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cgroup":{"memory":{"id":"session-c2.scope","mem":{"limit":{"bytes":9223372036854771712},"usage":{"bytes":631898112}}}},"cpu":{"system":{"ticks":70,"time":{"ms":70}},"total":{"ticks":260,"time":{"ms":263},"value":260},"user":{"ticks":190,"time":{"ms":193}}},"handles":{"limit":{"hard":1048576,"soft":1024},"open":10},"info":{"ephemeral_id":"d049bf8a-eb87-4810-9de6-4f9ceade755b","uptime":{"ms":34221},"version":"8.1.3"},"memstats":{"gc_next":21737728,"memory_alloc":12996064,"memory_sys":37569544,"memory_total":51874120,"rss":132845568},"runtime":{"goroutines":24}},"filebeat":{"events":{"active":6,"added":6,"done":0},"harvester":{"closed":0,"open_files":0,"running":0,"skipped":0,"started":0},"input":{"log":{"files":{"renamed":0,"truncated":0}},"netflow":{"flows":0,"packets":{"dropped":0,"received":0}}}},"libbeat":{"config":{"module":{"running":0,"starts":0,"stops":0},"reloads":1,"scans":1},"output":{"events":{"acked":0,"active":6,"batches":3,"dropped":0,"duplicates":0,"failed":12,"toomany":0,"total":18},"read":{"bytes":0,"errors":0},"type":"kafka","write":{"bytes":0,"errors":0}},"outputs":{"kafka":{"bytes_read":1130,"bytes_write":320}},"pipeline":{"clients":0,"events":{"active":6,"dropped":0,"failed":0,"filtered":0,"published":6,"retry":18,"total":6},"queue":{"acked":0,"max_events":4096}}},"registrar":{"states":{"cleanup":0,"current":0,"update":0},"writes":{"fail":0,"success":0,"total":0}},"system":{"cpu":{"cores":3},"load":{"1":0.04,"15":0,"5":0.01,"norm":{"1":0.0133,"15":0,"5":0.0033}}}},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-04-27T09:02:05.544+0200","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":193},"message":"Uptime: 34.22228469s","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:02:05.544+0200","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":160},"message":"Stopping metrics logging.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:02:05.545+0200","log.origin":{"file.name":"instance/beat.go","file.line":494},"message":"filebeat stopped.","service.name":"filebeat","ecs.version":"1.6.0"}

I have done another debug with filebeat -e -d "*", here is a thing that I have noticed

also I have done the command filebeat test output

Kafka: 10.60.101.11:9092...
  parse host... OK
  dns lookup... OK
  addresses: 10.60.101.11
  dial up... OK

The kafka-server is resolvable, the machine where filebeat runs also when I debug filbeat I can see that the connection with Kafka is established, it allow external connection to kafka, I have added the rules to my Fortimanger, and done some tests and it shouldn't block, also in the log of Forti thereis nothing saying that the connection is blocked.

CemG · April 27, 2022, 7:40am

If you need, there is the debug with the command filebeat -e -d "*"

{"log.level":"debug","@timestamp":"2022-04-27T09:29:46.755+0200","log.logger":"httpjson.transforms","log.origin":{"file.name":"httpjson/transform_registry.go","file.line":75},"message":"Register transform request:append","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:46.755+0200","log.logger":"httpjson.transforms","log.origin":{"file.name":"httpjson/transform_registry.go","file.line":75},"message":"Register transform request:delete","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:46.755+0200","log.logger":"httpjson.transforms","log.origin":{"file.name":"httpjson/transform_registry.go","file.line":75},"message":"Register transform request:set","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:46.755+0200","log.logger":"httpjson.transforms","log.origin":{"file.name":"httpjson/transform_registry.go","file.line":75},"message":"Register transform response:append","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:46.755+0200","log.logger":"httpjson.transforms","log.origin":{"file.name":"httpjson/transform_registry.go","file.line":75},"message":"Register transform response:delete","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:46.755+0200","log.logger":"httpjson.transforms","log.origin":{"file.name":"httpjson/transform_registry.go","file.line":75},"message":"Register transform response:set","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:46.755+0200","log.logger":"httpjson.transforms","log.origin":{"file.name":"httpjson/transform_registry.go","file.line":75},"message":"Register transform pagination:append","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:46.755+0200","log.logger":"httpjson.transforms","log.origin":{"file.name":"httpjson/transform_registry.go","file.line":75},"message":"Register transform pagination:delete","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:46.755+0200","log.logger":"httpjson.transforms","log.origin":{"file.name":"httpjson/transform_registry.go","file.line":75},"message":"Register transform pagination:set","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:46.755+0200","log.logger":"httpjson.transforms","log.origin":{"file.name":"httpjson/encoding.go","file.line":82},"message":"<nil>","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:46.755+0200","log.logger":"httpjson.transforms","log.origin":{"file.name":"httpjson/encoding.go","file.line":83},"message":"<nil>","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:46.755+0200","log.logger":"httpjson.transforms","log.origin":{"file.name":"httpjson/encoding.go","file.line":88},"message":"<nil>","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:46.755+0200","log.logger":"httpjson.transforms","log.origin":{"file.name":"httpjson/encoding.go","file.line":89},"message":"<nil>","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:46.755+0200","log.logger":"httpjson.transforms","log.origin":{"file.name":"httpjson/encoding.go","file.line":90},"message":"<nil>","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:29:46.756+0200","log.logger":"registrar","log.origin":{"file.name":"registrar/registrar.go","file.line":109},"message":"States Loaded from registrar: 0","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:29:46.756+0200","log.logger":"crawler","log.origin":{"file.name":"beater/crawler.go","file.line":71},"message":"Loading Inputs: 1","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:29:46.756+0200","log.logger":"crawler","log.origin":{"file.name":"beater/crawler.go","file.line":117},"message":"starting input, keys present on the config: [filebeat.inputs.0.enabled filebeat.inputs.0.id filebeat.inputs.0.paths.0 filebeat.inputs.0.type]","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:46.756+0200","log.logger":"registrar","log.origin":{"file.name":"registrar/registrar.go","file.line":140},"message":"Starting Registrar","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:46.756+0200","log.logger":"scanner","log.origin":{"file.name":"filestream/fswatch.go","file.line":278},"message":"recursive glob enabled","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:29:46.756+0200","log.logger":"crawler","log.origin":{"file.name":"beater/crawler.go","file.line":148},"message":"Starting input (ID: 7770775948050077450)","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:46.756+0200","log.logger":"cfgfile","log.origin":{"file.name":"cfgfile/reload.go","file.line":132},"message":"Checking module configs from: /etc/filebeat/modules.d/*.yml","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:29:46.756+0200","log.logger":"input.filestream","log.origin":{"file.name":"compat/compat.go","file.line":111},"message":"Input filestream starting","service.name":"filebeat","id":"apache-","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:46.756+0200","log.logger":"input.filestream","log.origin":{"file.name":"filestream/prospector.go","file.line":133},"message":"Starting prospector","service.name":"filebeat","id":"apache-","prospector":"file_prospector","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:29:46.756+0200","log.logger":"file_watcher","log.origin":{"file.name":"filestream/fswatch.go","file.line":138},"message":"Start next scan","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:46.756+0200","log.logger":"file_watcher","log.origin":{"file.name":"filestream/fswatch.go","file.line":205},"message":"Found 1 paths","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:46.756+0200","log.logger":"cfgfile","log.origin":{"file.name":"cfgfile/reload.go","file.line":146},"message":"Number of module configs found: 0","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:29:46.756+0200","log.logger":"crawler","log.origin":{"file.name":"beater/crawler.go","file.line":106},"message":"Loading and starting Inputs completed. Enabled inputs: 1","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:46.756+0200","log.logger":"input.filestream","log.origin":{"file.name":"filestream/prospector.go","file.line":180},"message":"A new file /var/log/apache2/access.log has been found","service.name":"filebeat","id":"apache-","prospector":"file_prospector","operation":"create","source_name":"native::523373-2054","os_id":"523373-2054","new_path":"/var/log/apache2/access.log","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:46.757+0200","log.logger":"input.filestream","log.origin":{"file.name":"input-logfile/harvester.go","file.line":145},"message":"Starting harvester for file","service.name":"filebeat","id":"apache-","source_file":"filestream::apache-::native::523373-2054","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:46.757+0200","log.logger":"input.filestream","log.origin":{"file.name":"filestream/input.go","file.line":174},"message":"newLogFileReader with config.MaxBytes:10485760","service.name":"filebeat","id":"apache-","source_file":"filestream::apache-::native::523373-2054","path":"/var/log/apache2/access.log","state-id":"native::523373-2054","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:29:46.757+0200","log.logger":"detect_null_bytes","log.origin":{"file.name":"debug/debug.go","file.line":95},"message":"Starting debug reader with a buffer size of 16384 and max failures of 100","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:46.757+0200","log.logger":"processors","log.origin":{"file.name":"processing/processors.go","file.line":209},"message":"Publish event: {\n  \"@timestamp\": \"2022-04-27T07:29:46.757Z\",\n  \"@metadata\": {\n    \"beat\": \"filebeat\",\n    \"type\": \"_doc\",\n    \"version\": \"8.1.3\"\n  },\n  \"log\": {\n    \"offset\": 0,\n    \"file\": {\n      \"path\": \"/var/log/apache2/access.log\"\n    }\n  },\n  \"message\": \"::1 - - [26/Apr/2022:13:46:29 +0200] \\\"GET / HTTP/1.1\\\" 200 10956 \\\"-\\\" \\\"curl/7.64.0\\\"\",\n  \"input\": {\n    \"type\": \"filestream\"\n  },\n  \"host\": {\n    \"name\": \"scanner\",\n    \"ip\": [\n      \"192.168.60.43\",\n      \"fe80::250:56ff:fea5:c4b5\"\n    ],\n    \"mac\": [\n      \"xxxxxx\"\n    ],\n    \"hostname\": \"scanner\",\n    \"architecture\": \"x86_64\",\n    \"os\": {\n      \"version\": \"10 (buster)\",\n      \"family\": \"debian\",\n      \"name\": \"Debian GNU/Linux\",\n      \"kernel\": \"4.19.0-20-amd64\",\n      \"codename\": \"buster\",\n      \"type\": \"linux\",\n      \"platform\": \"debian\"\n    },\n    \"id\": \"3d9b037af48a488884677d254db8ca99\",\n    \"containerized\": false\n  },\n  \"agent\": {\n    \"name\": \"scanner\",\n    \"type\": \"filebeat\",\n    \"version\": \"8.1.3\",\n    \"ephemeral_id\": \"1bf744f1-3bc8-485c-a770-29cc7952d428\",\n    \"id\": \"c37eb5db-c6f2-492c-a8a3-63d8aa695170\"\n  },\n  \"ecs\": {\n    \"version\": \"8.0.0\"\n  }\n}","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:46.757+0200","log.logger":"processors","log.origin":{"file.name":"processing/processors.go","file.line":209},"message":"Publish event: {\n  \"@timestamp\": \"2022-04-27T07:29:46.757Z\",\n  \"@metadata\": {\n    \"beat\": \"filebeat\",\n    \"type\": \"_doc\",\n    \"version\": \"8.1.3\"\n  },\n  \"host\": {\n    \"containerized\": false,\n    \"ip\": [\n      \"192.168.60.43\",\n      \"fe80::250:56ff:fea5:c4b5\"\n    ],\n    \"mac\": [\n      \"xxxxxxxxxxxxxxxx\"\n    ],\n    \"hostname\": \"scanner\",\n    \"architecture\": \"x86_64\",\n    \"name\": \"scanner\",\n    \"os\": {\n      \"codename\": \"buster\",\n      \"type\": \"linux\",\n      \"platform\": \"debian\",\n      \"version\": \"10 (buster)\",\n      \"family\": \"debian\",\n      \"name\": \"Debian GNU/Linux\",\n      \"kernel\": \"4.19.0-20-amd64\"\n    },\n    \"id\": \"3d9b037af48a488884677d254db8ca99\"\n  },\n  \"log\": {\n    \"offset\": 82,\n    \"file\": {\n      \"path\": \"/var/log/apache2/access.log\"\n    }\n  },\n  \"message\": \"::1 - - [26/Apr/2022:13:47:58 +0200] \\\"GET / HTTP/1.1\\\" 200 10956 \\\"-\\\" \\\"curl/7.64.0\\\"\",\n  \"input\": {\n    \"type\": \"filestream\"\n  },\n  \"agent\": {\n    \"type\": \"filebeat\",\n    \"version\": \"8.1.3\",\n    \"ephemeral_id\": \"1bf744f1-3bc8-485c-a770-29cc7952d428\",\n    \"id\": \"c37eb5db-c6f2-492c-a8a3-63d8aa695170\",\n    \"name\": \"scanner\"\n  },\n  \"ecs\": {\n    \"version\": \"8.0.0\"\n  }\n}","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:46.757+0200","log.logger":"processors","log.origin":{"file.name":"processing/processors.go","file.line":209},"message":"Publish event: {\n  \"@timestamp\": \"2022-04-27T07:29:46.757Z\",\n  \"@metadata\": {\n    \"beat\": \"filebeat\",\n    \"type\": \"_doc\",\n    \"version\": \"8.1.3\"\n  },\n  \"log\": {\n    \"offset\": 164,\n    \"file\": {\n      \"path\": \"/var/log/apache2/access.log\"\n    }\n  },\n  \"message\": \"::1 - - [26/Apr/2022:13:50:26 +0200] \\\"GET / HTTP/1.1\\\" 200 10956 \\\"-\\\" \\\"curl/7.64.0\\\"\",\n  \"input\": {\n    \"type\": \"filestream\"\n  },\n  \"host\": {\n    \"ip\": [\n      \"192.168.60.43\",\n      \"fe80::250:56ff:fea5:c4b5\"\n    ],\n    \"mac\": [\n      \"xxxxxxxxxxxxxxxx\"\n    ],\n    \"hostname\": \"scanner\",\n    \"architecture\": \"x86_64\",\n    \"name\": \"scanner\",\n    \"os\": {\n      \"family\": \"debian\",\n      \"name\": \"Debian GNU/Linux\",\n      \"kernel\": \"4.19.0-20-amd64\",\n      \"codename\": \"buster\",\n      \"type\": \"linux\",\n      \"platform\": \"debian\",\n      \"version\": \"10 (buster)\"\n    },\n    \"id\": \"3d9b037af48a488884677d254db8ca99\",\n    \"containerized\": false\n  },\n  \"agent\": {\n    \"id\": \"c37eb5db-c6f2-492c-a8a3-63d8aa695170\",\n    \"name\": \"scanner\",\n    \"type\": \"filebeat\",\n    \"version\": \"8.1.3\",\n    \"ephemeral_id\": \"1bf744f1-3bc8-485c-a770-29cc7952d428\"\n  },\n  \"ecs\": {\n    \"version\": \"8.0.0\"\n  }\n}","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:46.757+0200","log.logger":"processors","log.origin":{"file.name":"processing/processors.go","file.line":209},"message":"Publish event: {\n  \"@timestamp\": \"2022-04-27T07:29:46.757Z\",\n  \"@metadata\": {\n    \"beat\": \"filebeat\",\n    \"type\": \"_doc\",\n    \"version\": \"8.1.3\"\n  },\n  \"message\": \"::1 - - [26/Apr/2022:13:51:18 +0200] \\\"GET / HTTP/1.1\\\" 200 10956 \\\"-\\\" \\\"curl/7.64.0\\\"\",\n  \"input\": {\n    \"type\": \"filestream\"\n  },\n  \"host\": {\n    \"mac\": [\n      \"xxxxxxxxxxxxxxxxxx\"\n    ],\n    \"hostname\": \"scanner\",\n    \"name\": \"scanner\",\n    \"architecture\": \"x86_64\",\n    \"os\": {\n      \"name\": \"Debian GNU/Linux\",\n      \"kernel\": \"4.19.0-20-amd64\",\n      \"codename\": \"buster\",\n      \"type\": \"linux\",\n      \"platform\": \"debian\",\n      \"version\": \"10 (buster)\",\n      \"family\": \"debian\"\n    },\n    \"id\": \"3d9b037af48a488884677d254db8ca99\",\n    \"containerized\": false,\n    \"ip\": [\n      \"192.168.60.43\",\n      \"fe80::250:56ff:fea5:c4b5\"\n    ]\n  },\n  \"agent\": {\n    \"ephemeral_id\": \"1bf744f1-3bc8-485c-a770-29cc7952d428\",\n    \"id\": \"c37eb5db-c6f2-492c-a8a3-63d8aa695170\",\n    \"name\": \"scanner\",\n    \"type\": \"filebeat\",\n    \"version\": \"8.1.3\"\n  },\n  \"ecs\": {\n    \"version\": \"8.0.0\"\n  },\n  \"log\": {\n    \"file\": {\n      \"path\": \"/var/log/apache2/access.log\"\n    },\n    \"offset\": 246\n  }\n}","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:46.758+0200","log.logger":"processors","log.origin":{"file.name":"processing/processors.go","file.line":209},"message":"Publish event: {\n  \"@timestamp\": \"2022-04-27T07:29:46.757Z\",\n  \"@metadata\": {\n    \"beat\": \"filebeat\",\n    \"type\": \"_doc\",\n    \"version\": \"8.1.3\"\n  },\n  \"host\": {\n    \"name\": \"scanner\",\n    \"architecture\": \"x86_64\",\n    \"os\": {\n      \"codename\": \"buster\",\n      \"type\": \"linux\",\n      \"platform\": \"debian\",\n      \"version\": \"10 (buster)\",\n      \"family\": \"debian\",\n      \"name\": \"Debian GNU/Linux\",\n      \"kernel\": \"4.19.0-20-amd64\"\n    },\n    \"id\": \"3d9b037af48a488884677d254db8ca99\",\n    \"containerized\": false,\n    \"ip\": [\n      \"192.168.60.43\",\n      \"fe80::250:56ff:fea5:c4b5\"\n    ],\n    \"mac\": [\n      \"xxxxxxxxxxxxxxx\"\n    ],\n    \"hostname\": \"scanner\"\n  },\n  \"agent\": {\n    \"type\": \"filebeat\",\n    \"version\": \"8.1.3\",\n    \"ephemeral_id\": \"1bf744f1-3bc8-485c-a770-29cc7952d428\",\n    \"id\": \"c37eb5db-c6f2-492c-a8a3-63d8aa695170\",\n    \"name\": \"scanner\"\n  },\n  \"ecs\": {\n    \"version\": \"8.0.0\"\n  },\n  \"log\": {\n    \"offset\": 328,\n    \"file\": {\n      \"path\": \"/var/log/apache2/access.log\"\n    }\n  },\n  \"message\": \"::1 - - [26/Apr/2022:14:27:23 +0200] \\\"GET / HTTP/1.1\\\" 200 10956 \\\"-\\\" \\\"curl/7.64.0\\\"\",\n  \"input\": {\n    \"type\": \"filestream\"\n  }\n}","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:46.758+0200","log.logger":"processors","log.origin":{"file.name":"processing/processors.go","file.line":209},"message":"Publish event: {\n  \"@timestamp\": \"2022-04-27T07:29:46.758Z\",\n  \"@metadata\": {\n    \"beat\": \"filebeat\",\n    \"type\": \"_doc\",\n    \"version\": \"8.1.3\"\n  },\n  \"log\": {\n    \"offset\": 410,\n    \"file\": {\n      \"path\": \"/var/log/apache2/access.log\"\n    }\n  },\n  \"message\": \"::1 - - [26/Apr/2022:16:17:35 +0200] \\\"GET / HTTP/1.1\\\" 200 10956 \\\"-\\\" \\\"curl/7.64.0\\\"\",\n  \"input\": {\n    \"type\": \"filestream\"\n  },\n  \"ecs\": {\n    \"version\": \"8.0.0\"\n  },\n  \"host\": {\n    \"hostname\": \"scanner\",\n    \"name\": \"scanner\",\n    \"architecture\": \"x86_64\",\n    \"os\": {\n      \"platform\": \"debian\",\n      \"version\": \"10 (buster)\",\n      \"family\": \"debian\",\n      \"name\": \"Debian GNU/Linux\",\n      \"kernel\": \"4.19.0-20-amd64\",\n      \"codename\": \"buster\",\n      \"type\": \"linux\"\n    },\n    \"id\": \"3d9b037af48a488884677d254db8ca99\",\n    \"containerized\": false,\n    \"ip\": [\n      \"192.168.60.43\",\n      \"fe80::250:56ff:fea5:c4b5\"\n    ],\n    \"mac\": [\n      \"xxxxxxxxxxxxx\"\n    ]\n  },\n  \"agent\": {\n    \"name\": \"scanner\",\n    \"type\": \"filebeat\",\n    \"version\": \"8.1.3\",\n    \"ephemeral_id\": \"1bf744f1-3bc8-485c-a770-29cc7952d428\",\n    \"id\": \"c37eb5db-c6f2-492c-a8a3-63d8aa695170\"\n  }\n}","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:46.758+0200","log.logger":"input.filestream","log.origin":{"file.name":"filestream/filestream.go","file.line":131},"message":"End of file reached: /var/log/apache2/access.log; Backoff now.","service.name":"filebeat","id":"apache-","source_file":"filestream::apache-::native::523373-2054","path":"/var/log/apache2/access.log","state-id":"native::523373-2054","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:29:46.756+0200","log.origin":{"file.name":"cfgfile/reload.go","file.line":164},"message":"Config reloader started","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:46.758+0200","log.logger":"cfgfile","log.origin":{"file.name":"cfgfile/reload.go","file.line":194},"message":"Scan for new config files","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:46.758+0200","log.logger":"cfgfile","log.origin":{"file.name":"cfgfile/reload.go","file.line":213},"message":"Number of module configs found: 0","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:46.758+0200","log.logger":"reload","log.origin":{"file.name":"cfgfile/list.go","file.line":63},"message":"Starting reload procedure, current runners: 0","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:46.758+0200","log.logger":"reload","log.origin":{"file.name":"cfgfile/list.go","file.line":81},"message":"Start list: 0, Stop list: 0","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:29:46.758+0200","log.origin":{"file.name":"cfgfile/reload.go","file.line":224},"message":"Loading of config files completed.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:29:47.759+0200","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":139},"message":"Connecting to kafka(10.60.101.11:9092)","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:47.759+0200","log.logger":"kafka","log.origin":{"file.name":"kafka/client.go","file.line":100},"message":"connect: [10.60.101.11:9092]","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:29:47.759+0200","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":147},"message":"Connection to kafka(10.60.101.11:9092) established","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:48.759+0200","log.logger":"input.filestream","log.origin":{"file.name":"filestream/filestream.go","file.line":131},"message":"End of file reached: /var/log/apache2/access.log; Backoff now.","service.name":"filebeat","id":"apache-","source_file":"filestream::apache-::native::523373-2054","path":"/var/log/apache2/access.log","state-id":"native::523373-2054","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:52.759+0200","log.logger":"input.filestream","log.origin":{"file.name":"filestream/filestream.go","file.line":131},"message":"End of file reached: /var/log/apache2/access.log; Backoff now.","service.name":"filebeat","id":"apache-","source_file":"filestream::apache-::native::523373-2054","path":"/var/log/apache2/access.log","state-id":"native::523373-2054","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:29:56.757+0200","log.logger":"file_watcher","log.origin":{"file.name":"filestream/fswatch.go","file.line":138},"message":"Start next scan","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:56.757+0200","log.logger":"file_watcher","log.origin":{"file.name":"filestream/fswatch.go","file.line":205},"message":"Found 1 paths","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:58.645+0200","log.logger":"kafka","log.origin":{"file.name":"kafka/client.go","file.line":372},"message":"finished kafka batch","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:58.645+0200","log.logger":"kafka","log.origin":{"file.name":"kafka/client.go","file.line":386},"message":"Kafka publish failed with: dial tcp: lookup interlog on 192.168.60.4:53: server misbehaving","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:58.645+0200","log.logger":"kafka","log.origin":{"file.name":"kafka/client.go","file.line":177},"message":"got event.Meta[\"partition\"] = 0","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:58.645+0200","log.logger":"kafka","log.origin":{"file.name":"kafka/client.go","file.line":177},"message":"got event.Meta[\"partition\"] = 0","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:58.645+0200","log.logger":"kafka","log.origin":{"file.name":"kafka/client.go","file.line":177},"message":"got event.Meta[\"partition\"] = 0","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:58.646+0200","log.logger":"kafka","log.origin":{"file.name":"kafka/client.go","file.line":177},"message":"got event.Meta[\"partition\"] = 0","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:58.646+0200","log.logger":"kafka","log.origin":{"file.name":"kafka/client.go","file.line":177},"message":"got event.Meta[\"partition\"] = 0","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:29:58.646+0200","log.logger":"kafka","log.origin":{"file.name":"kafka/client.go","file.line":177},"message":"got event.Meta[\"partition\"] = 0","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:30:00.760+0200","log.logger":"input.filestream","log.origin":{"file.name":"filestream/filestream.go","file.line":131},"message":"End of file reached: /var/log/apache2/access.log; Backoff now.","service.name":"filebeat","id":"apache-","source_file":"filestream::apache-::native::523373-2054","path":"/var/log/apache2/access.log","state-id":"native::523373-2054","ecs.version":"1.6.0"}
^C{"log.level":"debug","@timestamp":"2022-04-27T09:30:03.050+0200","log.logger":"service","log.origin":{"file.name":"service/service.go","file.line":54},"message":"Received sigterm/sigint, stopping","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:30:03.050+0200","log.origin":{"file.name":"beater/filebeat.go","file.line":427},"message":"Stopping filebeat","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:30:03.051+0200","log.origin":{"file.name":"beater/crawler.go","file.line":155},"message":"Stopping Crawler","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:30:03.052+0200","log.origin":{"file.name":"beater/crawler.go","file.line":165},"message":"Stopping 1 inputs","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:30:03.052+0200","log.origin":{"file.name":"cfgfile/reload.go","file.line":227},"message":"Dynamic config reloader stopped","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:30:03.052+0200","log.logger":"crawler","log.origin":{"file.name":"beater/crawler.go","file.line":170},"message":"Stopping input: 7770775948050077450","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:30:03.054+0200","log.logger":"input.filestream","log.origin":{"file.name":"compat/compat.go","file.line":132},"message":"Input 'filestream' stopped","service.name":"filebeat","id":"apache-","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:30:03.054+0200","log.logger":"input.filestream","log.origin":{"file.name":"filestream/input.go","file.line":143},"message":"Closing reader of filestream","service.name":"filebeat","id":"apache-","source_file":"filestream::apache-::native::523373-2054","path":"/var/log/apache2/access.log","state-id":"native::523373-2054","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:30:03.054+0200","log.logger":"publisher","log.origin":{"file.name":"pipeline/client.go","file.line":158},"message":"client: closing acker","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:30:03.054+0200","log.logger":"publisher","log.origin":{"file.name":"pipeline/client.go","file.line":163},"message":"client: done closing acker","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:30:03.054+0200","log.logger":"publisher","log.origin":{"file.name":"pipeline/client.go","file.line":165},"message":"client: unlink from queue","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:30:03.054+0200","log.logger":"publisher","log.origin":{"file.name":"pipeline/client.go","file.line":187},"message":"client: cancelled 0 events","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:30:03.055+0200","log.logger":"publisher","log.origin":{"file.name":"pipeline/client.go","file.line":167},"message":"client: done unlink","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:30:03.055+0200","log.logger":"publisher","log.origin":{"file.name":"pipeline/client.go","file.line":170},"message":"client: closing processors","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:30:03.055+0200","log.logger":"publisher","log.origin":{"file.name":"pipeline/client.go","file.line":175},"message":"client: done closing processors","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:30:03.054+0200","log.logger":"input.filestream","log.origin":{"file.name":"filestream/input.go","file.line":320},"message":"Reader was closed. Closing.","service.name":"filebeat","id":"apache-","source_file":"filestream::apache-::native::523373-2054","path":"/var/log/apache2/access.log","state-id":"native::523373-2054","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:30:03.055+0200","log.logger":"input.filestream","log.origin":{"file.name":"input-logfile/harvester.go","file.line":219},"message":"Stopped harvester for file","service.name":"filebeat","id":"apache-","source_file":"filestream::apache-::native::523373-2054","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:30:03.055+0200","log.logger":"input.filestream","log.origin":{"file.name":"filestream/prospector.go","file.line":165},"message":"Prospector has stopped","service.name":"filebeat","id":"apache-","prospector":"file_prospector","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:30:03.055+0200","log.logger":"input.filestream","log.origin":{"file.name":"compat/compat.go","file.line":124},"message":"Input 'filestream' stopped","service.name":"filebeat","id":"apache-","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:30:03.054+0200","log.origin":{"file.name":"beater/crawler.go","file.line":185},"message":"Crawler stopped","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:30:03.055+0200","log.logger":"registrar","log.origin":{"file.name":"registrar/registrar.go","file.line":132},"message":"Stopping Registrar","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:30:03.055+0200","log.logger":"registrar","log.origin":{"file.name":"registrar/registrar.go","file.line":166},"message":"Ending Registrar","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-04-27T09:30:03.055+0200","log.logger":"registrar","log.origin":{"file.name":"registrar/registrar.go","file.line":167},"message":"Stopping Registrar","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:30:03.055+0200","log.logger":"registrar","log.origin":{"file.name":"registrar/registrar.go","file.line":137},"message":"Registrar stopped","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:30:03.063+0200","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":192},"message":"Total metrics","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cgroup":{"memory":{"id":"session-c2.scope","mem":{"limit":{"bytes":9223372036854771712},"usage":{"bytes":631730176}}}},"cpu":{"system":{"ticks":40,"time":{"ms":49}},"total":{"ticks":200,"time":{"ms":214},"value":200},"user":{"ticks":160,"time":{"ms":165}}},"handles":{"limit":{"hard":1048576,"soft":1024},"open":10},"info":{"ephemeral_id":"1bf744f1-3bc8-485c-a770-29cc7952d428","uptime":{"ms":16373},"version":"8.1.3"},"memstats":{"gc_next":22809376,"memory_alloc":17420464,"memory_sys":33113096,"memory_total":51494840,"rss":128786432},"runtime":{"goroutines":24}},"filebeat":{"events":{"active":6,"added":6,"done":0},"harvester":{"closed":0,"open_files":0,"running":0,"skipped":0,"started":0},"input":{"log":{"files":{"renamed":0,"truncated":0}},"netflow":{"flows":0,"packets":{"dropped":0,"received":0}}}},"libbeat":{"config":{"module":{"running":0,"starts":0,"stops":0},"reloads":1,"scans":1},"output":{"events":{"acked":0,"active":6,"batches":2,"dropped":0,"duplicates":0,"failed":6,"toomany":0,"total":12},"read":{"bytes":0,"errors":0},"type":"kafka","write":{"bytes":0,"errors":0}},"outputs":{"kafka":{"bytes_read":565,"bytes_write":160}},"pipeline":{"clients":0,"events":{"active":6,"dropped":0,"failed":0,"filtered":0,"published":6,"retry":12,"total":6},"queue":{"acked":0,"max_events":4096}}},"registrar":{"states":{"cleanup":0,"current":0,"update":0},"writes":{"fail":0,"success":0,"total":0}},"system":{"cpu":{"cores":3},"load":{"1":0.02,"15":0,"5":0.02,"norm":{"1":0.0067,"15":0,"5":0.0067}}}},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-04-27T09:30:03.064+0200","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":193},"message":"Uptime: 16.381242707s","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:30:03.064+0200","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":160},"message":"Stopping metrics logging.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-27T09:30:03.064+0200","log.origin":{"file.name":"instance/beat.go","file.line":494},"message":"filebeat stopped.","service.name":"filebeat","ecs.version":"1.6.0"}

Andrea_Spacca · April 27, 2022, 7:54am

It seems to be this problem: Why Can’t I Connect to Kafka? | Troubleshoot Connectivity

filebeat test output tests only dialing to the defined output.kafka.hosts without brokers connection

CemG · April 27, 2022, 8:23am

Thanks I'll check it and say if it worked

CemG · April 27, 2022, 9:41am

@Andrea_Spacca Thank you that was the issue, it works now
Have a nice day

system · May 25, 2022, 11:42am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Filebeat to kafka failure Beats filebeat	5	422	May 13, 2021
Problem with sending logs from filebeat to logstash when I add kafka to pipeline Beats filebeat	3	514	December 3, 2020
Filebeat wont send all messages to kafka Beats filebeat	1	290	August 1, 2019
Filebeat is unable to send logs into kafka Beats filebeat	5	2109	November 9, 2020
Can filebeat monitoring send data to Kafka? Beats filebeat	8	675	October 26, 2019

Filebeat is connected to Kafka, but it doesn't send Log

Related topics