Fortinet Logs Integration

Hello,

I've installed the Fortinet Logs Integration and configured the Fortigate to send syslogs for 9004. I was able to test that the Fortigate is sending logs but nothing shows up in Elastic. Is there something I am missing? It doesn't look like anything is listening on the 9004 port either. I'm kinda stuck here, not sure where to go now. Any help would be greatly appreciated.

Hi @jumpie Welcome to the community.,, we are going to need a little more information...

How did you install filebeat?

Did you follow all these steps exactly but with the fortinet module?

Please post your filebeat.yml and the fortinet.yml

How are you starting filebeat?

Did you look at the filebeat logs? Are there any errors?

Thanks for replying @stephenb !

I checked the logs for Filebeat and don't see much. I did follow the steps you linked and filebeat seems to be running. Below is the requested configs and the log.

Log:

"log.level":"info","@timestamp":"2022-05-03T14:49:47.537Z","log.origin":{"file.name":"instance/beat.go","file.line":685},"message":"Home path: [/home/bottlebill/filebeat-8.2.0-linux-x86_64] Config path: [/home/bottlebill/filebeat-8.2.0-linux-x86_64] Data path: [/home/bottlebill/filebeat-8.2.0-linux-x86_64/data] Logs path: [/home/bottlebill/filebeat-8.2.0-linux-x86_64/logs]","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T14:49:47.537Z","log.origin":{"file.name":"instance/beat.go","file.line":693},"message":"Beat ID: d309f519-a6ba-4880-9037-e13269ca7dd7","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2022-05-03T14:49:50.540Z","log.logger":"add_cloud_metadata","log.origin":{"file.name":"add_cloud_metadata/provider_aws_ec2.go","file.line":80},"message":"read token request for getting IMDSv2 token returns empty: Put \"http://169.254.169.254/latest/api/token\": context deadline exceeded (Client.Timeout exceeded while awaiting headers). No token in the metadata request will be used.","service.name":"filebeat","ecs.version":"1.6.0"}

filebeat.yml

filebeat.inputs:

- type: filestream
  id: my-filestream-id
  enabled: false
  paths:
    - /var/log/*.log
	
filebeat.config.modules:  
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false

setup.template.settings:
  index.number_of_shards: 1

	
setup.kibana:

  
   host: "10.10.10.10:5601"
   username: "user"
   password: "password"
  
output.elasticsearch:
   hosts: ["https://localhost:9200"]
   ssl.verification_mode: none
   username: "user"
   password: "password"


processors:
  - add_host_metadata:
      when.not.contains.tags: forwarded
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - add_kubernetes_metadata: ~

fortinet.yml

- module: fortinet
  firewall:
    enabled: true

 clientendpoint:
    enabled: false

 fortimail:
    enabled: false

 fortimanager:
    enabled: false

In the future please format your code... using the </> button above (i did that because I could not read it well :wink: )

What version are you on? 8.2?

Not sure "what don't see much means"

Please try to be a bit more specific... like

I went to discover in Kibana and I did not see any logs under the filebeat-* index pattern... is that what you mean?

Make sure your forinet is sending to the correct ip address and port? and have the correct interface.

So perhaps set the setting that will work with your expected config... example

- module: fortinet
  firewall:
    enabled: true
    var.input: udp
    var.syslog_host: 0.0.0.0
    var.syslog_port: 9004

You should also see a log lines like the following in the filebeat startup... there are many logs on startup 50 or so lines..

{"log.level":"info","@timestamp":"2022-05-03T09:51:08.022-0700","log.logger":"modules","log.origin":{"file.name":"fileset/pipelines.go","file.line":133},"message":"Elasticsearch pipeline loaded.","service.name":"filebeat","pipeline":"filebeat-8.1.2-fortinet-firewall-pipeline","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:51:08.121-0700","log.logger":"modules","log.origin":{"file.name":"fileset/pipelines.go","file.line":133},"message":"Elasticsearch pipeline loaded.","service.name":"filebeat","pipeline":"filebeat-8.1.2-fortinet-firewall-event","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:51:08.213-0700","log.logger":"modules","log.origin":{"file.name":"fileset/pipelines.go","file.line":133},"message":"Elasticsearch pipeline loaded.","service.name":"filebeat","pipeline":"filebeat-8.1.2-fortinet-firewall-utm","ecs.version":"1.6.0"}
....
{"log.level":"info","@timestamp":"2022-05-03T09:51:08.297-0700","log.origin":{"file.name":"udp/input.go","file.line":99},"message":"Starting UDP input","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:51:08.297-0700","log.logger":"UDP","log.origin":{"file.name":"dgram/server.go","file.line":99},"message":"Started listening for UDP connection","service.name":"filebeat","ecs.version":"1.6.0"}

Okay, sorry. I'll use the </> from now on. I'm using Filebeat 8.2.0. The comment in fortinet.yml above the syslog_port sayd it defaults to 9004. I did add the fields like in the example to the fortinet.yml. Not seeing any syslogs coming through though and the nothing is listening on port 9004.

Filebeat log from start:

user@es01:~/filebeat-8.2.0-linux-x86_64$ sudo ./filebeat -e
{"log.level":"info","@timestamp":"2022-05-03T09:52:26.216-0700","log.origin":{"file.name":"instance/beat.go","file.line":685},"message":"Home path: [/home/user/filebeat-8.2.0-linux-x86_64] Config path: [/home/user/filebeat-8.2.0-linux-x86_64] Data path: [/home/user/filebeat-8.2.0-linux-x86_64/data] Logs path: [/home/user/filebeat-8.2.0-linux-x86_64/logs]","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:52:26.216-0700","log.origin":{"file.name":"instance/beat.go","file.line":693},"message":"Beat ID: d309f519-a6ba-4880-9037-e13269ca7dd7","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2022-05-03T09:52:29.219-0700","log.logger":"add_cloud_metadata","log.origin":{"file.name":"add_cloud_metadata/provider_aws_ec2.go","file.line":80},"message":"read token request for getting IMDSv2 token returns empty: Put \"http://169.254.169.254/latest/api/token\": context deadline exceeded (Client.Timeout exceeded while awaiting headers). No token in the metadata request will be used.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:52:29.221-0700","log.logger":"seccomp","log.origin":{"file.name":"seccomp/seccomp.go","file.line":124},"message":"Syscall filter successfully installed","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:52:29.221-0700","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1063},"message":"Beat info","service.name":"filebeat","system_info":{"beat":{"path":{"config":"/home/user/filebeat-8.2.0-linux-x86_64","data":"/home/user/filebeat-8.2.0-linux-x86_64/data","home":"/home/user/filebeat-8.2.0-linux-x86_64","logs":"/home/user/filebeat-8.2.0-linux-x86_64/logs"},"type":"filebeat","uuid":"d309f519-a6ba-4880-9037-e13269ca7dd7"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-05-03T09:52:29.221-0700","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1072},"message":"Build info","service.name":"filebeat","system_info":{"build":{"commit":"045da3a1bb89944373c33332c18ca99ef6192df2","libbeat":"8.2.0","time":"2022-04-19T23:31:06.000Z","version":"8.2.0"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-05-03T09:52:29.221-0700","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1075},"message":"Go runtime info","service.name":"filebeat","system_info":{"go":{"os":"linux","arch":"amd64","max_procs":6,"version":"go1.17.8"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-05-03T09:52:29.221-0700","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1079},"message":"Host info","service.name":"filebeat","system_info":{"host":{"architecture":"x86_64","boot_time":"2022-05-02T14:25:09-07:00","containerized":false,"name":"es01","ip":["127.0.0.1/8","::1/128","10.20.30.115/24","fe80::215:5dff:fe04:d24/64"],"kernel_version":"5.4.0-109-generic","mac":["00:15:5d:04:0d:24"],"os":{"type":"linux","family":"debian","platform":"ubuntu","name":"Ubuntu","version":"20.04.4 LTS (Focal Fossa)","major":20,"minor":4,"patch":4,"codename":"focal"},"timezone":"PDT","timezone_offset_sec":-25200,"id":"721ec161f76e4baf9de749a6e09b391d"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-05-03T09:52:29.222-0700","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1108},"message":"Process info","service.name":"filebeat","system_info":{"process":{"capabilities":{"inheritable":null,"permitted":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"effective":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"bounding":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"ambient":null},"cwd":"/home/user/filebeat-8.2.0-linux-x86_64","exe":"/home/user/filebeat-8.2.0-linux-x86_64/filebeat","name":"filebeat","pid":9921,"ppid":9920,"seccomp":{"mode":"filter","no_new_privs":true},"start_time":"2022-05-03T09:52:25.470-0700"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-05-03T09:52:29.222-0700","log.origin":{"file.name":"instance/beat.go","file.line":325},"message":"Setup Beat: filebeat; Version: 8.2.0","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2022-05-03T09:52:29.223-0700","log.logger":"cfgwarn","log.origin":{"file.name":"tlscommon/config.go","file.line":102},"message":"DEPRECATED: Treating the CommonName field on X.509 certificates as a host name when no Subject Alternative Names are present is going to be removed. Please update your certificates if needed. Will be removed in version: 8.0.0","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:52:29.223-0700","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":105},"message":"elasticsearch url: https://localhost:9200","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2022-05-03T09:52:29.223-0700","log.logger":"tls","log.origin":{"file.name":"tlscommon/tls_config.go","file.line":105},"message":"SSL/TLS verifications disabled.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:52:29.223-0700","log.logger":"publisher","log.origin":{"file.name":"pipeline/module.go","file.line":113},"message":"Beat name: es01","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:52:29.223-0700","log.logger":"modules","log.origin":{"file.name":"fileset/modules.go","file.line":108},"message":"Enabled modules/filesets: ","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:52:29.223-0700","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":142},"message":"Starting metrics logging every 30s","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:52:29.223-0700","log.origin":{"file.name":"instance/beat.go","file.line":505},"message":"filebeat start running.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:52:29.224-0700","log.origin":{"file.name":"memlog/store.go","file.line":134},"message":"Finished loading transaction log file for '/home/user/filebeat-8.2.0-linux-x86_64/data/registry/filebeat'. Active transaction id=0","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:52:29.224-0700","log.logger":"registrar","log.origin":{"file.name":"registrar/registrar.go","file.line":109},"message":"States Loaded from registrar: 0","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:52:29.224-0700","log.logger":"crawler","log.origin":{"file.name":"beater/crawler.go","file.line":71},"message":"Loading Inputs: 1","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:52:29.224-0700","log.logger":"crawler","log.origin":{"file.name":"beater/crawler.go","file.line":117},"message":"starting input, keys present on the config: [filebeat.inputs.0.enabled filebeat.inputs.0.id filebeat.inputs.0.paths.0 filebeat.inputs.0.type]","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:52:29.224-0700","log.logger":"crawler","log.origin":{"file.name":"beater/crawler.go","file.line":121},"message":"input disabled, skipping it","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:52:29.226-0700","log.logger":"modules","log.origin":{"file.name":"fileset/modules.go","file.line":108},"message":"Enabled modules/filesets: fortinet (firewall)","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:52:29.226-0700","log.logger":"crawler","log.origin":{"file.name":"beater/crawler.go","file.line":106},"message":"Loading and starting Inputs completed. Enabled inputs: 0","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:52:29.226-0700","log.origin":{"file.name":"cfgfile/reload.go","file.line":164},"message":"Config reloader started","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:52:29.227-0700","log.logger":"modules","log.origin":{"file.name":"fileset/modules.go","file.line":108},"message":"Enabled modules/filesets: fortinet (firewall)","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:52:29.228-0700","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":105},"message":"elasticsearch url: https://localhost:9200","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2022-05-03T09:52:29.228-0700","log.logger":"tls","log.origin":{"file.name":"tlscommon/tls_config.go","file.line":105},"message":"SSL/TLS verifications disabled.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2022-05-03T09:52:29.228-0700","log.logger":"tls","log.origin":{"file.name":"tlscommon/tls_config.go","file.line":105},"message":"SSL/TLS verifications disabled.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:52:29.256-0700","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":287},"message":"Attempting to connect to Elasticsearch version 8.1.3","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:52:29.265-0700","log.origin":{"file.name":"cfgfile/reload.go","file.line":224},"message":"Loading of config files completed.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:52:29.265-0700","log.origin":{"file.name":"udp/input.go","file.line":98},"message":"Starting UDP input","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:52:29.265-0700","log.logger":"UDP","log.origin":{"file.name":"dgram/server.go","file.line":99},"message":"Started listening for UDP connection","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:52:32.221-0700","log.logger":"add_cloud_metadata","log.origin":{"file.name":"add_cloud_metadata/add_cloud_metadata.go","file.line":101},"message":"add_cloud_metadata: hosting provider type not detected.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:52:37.747-0700","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":139},"message":"Connecting to backoff(elasticsearch(https://localhost:9200))","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2022-05-03T09:52:37.748-0700","log.logger":"tls","log.origin":{"file.name":"tlscommon/tls_config.go","file.line":105},"message":"SSL/TLS verifications disabled.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:52:37.775-0700","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":287},"message":"Attempting to connect to Elasticsearch version 8.1.3","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:52:37.781-0700","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":287},"message":"Attempting to connect to Elasticsearch version 8.1.3","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2022-05-03T09:52:39.574-0700","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":150},"message":"Failed to connect to backoff(elasticsearch(https://localhost:9200)): Connection marked as failed because the onConnect callback failed: Elasticsearch is too old. Please upgrade the instance. If you would like to connect to older instances set output.elasticsearch.allow_older_versions to true. ES=8.1.3, Beat=8.2.0.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:52:39.575-0700","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":141},"message":"Attempting to reconnect to backoff(elasticsearch(https://localhost:9200)) with 1 reconnect attempt(s)","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:52:39.576-0700","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":287},"message":"Attempting to connect to Elasticsearch version 8.1.3","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2022-05-03T09:52:41.990-0700","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":150},"message":"Failed to connect to backoff(elasticsearch(https://localhost:9200)): Connection marked as failed because the onConnect callback failed: Elasticsearch is too old. Please upgrade the instance. If you would like to connect to older instances set output.elasticsearch.allow_older_versions to true. ES=8.1.3, Beat=8.2.0.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:52:41.990-0700","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":141},"message":"Attempting to reconnect to backoff(elasticsearch(https://localhost:9200)) with 2 reconnect attempt(s)","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:52:41.991-0700","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":287},"message":"Attempting to connect to Elasticsearch version 8.1.3","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2022-05-03T09:52:46.800-0700","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":150},"message":"Failed to connect to backoff(elasticsearch(https://localhost:9200)): Connection marked as failed because the onConnect callback failed: Elasticsearch is too old. Please upgrade the instance. If you would like to connect to older instances set output.elasticsearch.allow_older_versions to true. ES=8.1.3, Beat=8.2.0.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:52:46.800-0700","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":141},"message":"Attempting to reconnect to backoff(elasticsearch(https://localhost:9200)) with 3 reconnect attempt(s)","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:52:46.801-0700","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":287},"message":"Attempting to connect to Elasticsearch version 8.1.3","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2022-05-03T09:52:58.582-0700","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":150},"message":"Failed to connect to backoff(elasticsearch(https://localhost:9200)): Connection marked as failed because the onConnect callback failed: Elasticsearch is too old. Please upgrade the instance. If you would like to connect to older instances set output.elasticsearch.allow_older_versions to true. ES=8.1.3, Beat=8.2.0.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:52:58.582-0700","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":141},"message":"Attempting to reconnect to backoff(elasticsearch(https://localhost:9200)) with 4 reconnect attempt(s)","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:52:58.583-0700","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":287},"message":"Attempting to connect to Elasticsearch version 8.1.3","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:52:59.227-0700","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":184},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cgroup":{"cpu":{"cfs":{"period":{"us":100000}},"id":"user.slice"},"cpuacct":{"id":"user.slice","total":{"ns":8308597454590}},"memory":{"id":"session-8.scope","mem":{"limit":{"bytes":9223372036854771712},"usage":{"bytes":702971904}}}},"cpu":{"system":{"ticks":280,"time":{"ms":280}},"total":{"ticks":840,"time":{"ms":840},"value":0},"user":{"ticks":560,"time":{"ms":560}}},"handles":{"limit":{"hard":1048576,"soft":1024},"open":13},"info":{"ephemeral_id":"14c8ce41-c547-46b8-8cf1-e6178435730b","uptime":{"ms":33087},"version":"8.2.0"},"memstats":{"gc_next":56125168,"memory_alloc":33099736,"memory_sys":59327496,"memory_total":145374544,"rss":153948160},"runtime":{"goroutines":36}},"filebeat":{"events":{"active":4104,"added":4104},"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":1,"starts":1},"reloads":1,"scans":1},"output":{"events":{"active":0},"read":{"bytes":5702},"type":"elasticsearch","write":{"bytes":3050}},"pipeline":{"clients":1,"events":{"active":4100,"published":4100,"retry":2004,"total":4100},"queue":{"max_events":4096}}},"registrar":{"states":{"current":0}},"system":{"cpu":{"cores":6},"load":{"1":0.09,"15":0.14,"5":0.2,"norm":{"1":0.015,"15":0.0233,"5":0.0333}}}},"ecs.version":"1.6.0"}}
{"log.level":"error","@timestamp":"2022-05-03T09:53:15.219-0700","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":150},"message":"Failed to connect to backoff(elasticsearch(https://localhost:9200)): Connection marked as failed because the onConnect callback failed: Elasticsearch is too old. Please upgrade the instance. If you would like to connect to older instances set output.elasticsearch.allow_older_versions to true. ES=8.1.3, Beat=8.2.0.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:53:15.220-0700","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":141},"message":"Attempting to reconnect to backoff(elasticsearch(https://localhost:9200)) with 5 reconnect attempt(s)","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:53:15.221-0700","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":287},"message":"Attempting to connect to Elasticsearch version 8.1.3","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:53:29.226-0700","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":184},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cgroup":{"cpuacct":{"total":{"ns":3171815859}},"memory":{"mem":{"usage":{"bytes":-98304}}}},"cpu":{"system":{"ticks":280},"total":{"ticks":850,"time":{"ms":10},"value":0},"user":{"ticks":570,"time":{"ms":10}}},"handles":{"limit":{"hard":1048576,"soft":1024},"open":13},"info":{"ephemeral_id":"14c8ce41-c547-46b8-8cf1-e6178435730b","uptime":{"ms":63086},"version":"8.2.0"},"memstats":{"gc_next":56125168,"memory_alloc":33815080,"memory_total":146089888,"rss":153948160},"runtime":{"goroutines":36}},"filebeat":{"events":{"active":13,"added":13},"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":1}},"output":{"events":{"active":0},"read":{"bytes":633},"write":{"bytes":297}},"pipeline":{"clients":1,"events":{"active":4117,"published":16,"retry":501,"total":17}}},"registrar":{"states":{"current":0}},"system":{"load":{"1":0.06,"15":0.14,"5":0.18,"norm":{"1":0.01,"15":0.0233,"5":0.03}}}},"ecs.version":"1.6.0"}}
{"log.level":"error","@timestamp":"2022-05-03T09:53:48.692-0700","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":150},"message":"Failed to connect to backoff(elasticsearch(https://localhost:9200)): Connection marked as failed because the onConnect callback failed: Elasticsearch is too old. Please upgrade the instance. If you would like to connect to older instances set output.elasticsearch.allow_older_versions to true. ES=8.1.3, Beat=8.2.0.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:53:48.692-0700","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":141},"message":"Attempting to reconnect to backoff(elasticsearch(https://localhost:9200)) with 6 reconnect attempt(s)","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:53:48.693-0700","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":287},"message":"Attempting to connect to Elasticsearch version 8.1.3","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:53:59.226-0700","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":184},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cgroup":{"cpuacct":{"total":{"ns":6139397272}},"memory":{"mem":{"usage":{"bytes":-102400}}}},"cpu":{"system":{"ticks":290,"time":{"ms":10}},"total":{"ticks":860,"time":{"ms":10},"value":0},"user":{"ticks":570}},"handles":{"limit":{"hard":1048576,"soft":1024},"open":12},"info":{"ephemeral_id":"14c8ce41-c547-46b8-8cf1-e6178435730b","uptime":{"ms":93086},"version":"8.2.0"},"memstats":{"gc_next":56125168,"memory_alloc":34156192,"memory_total":146431000,"rss":153948160},"runtime":{"goroutines":34}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":1}},"output":{"events":{"active":0},"read":{"bytes":1109},"write":{"bytes":614}},"pipeline":{"clients":1,"events":{"active":4117,"retry":501}}},"registrar":{"states":{"current":0}},"system":{"load":{"1":0.03,"15":0.13,"5":0.16,"norm":{"1":0.005,"15":0.0217,"5":0.0267}}}},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-05-03T09:54:29.226-0700","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":184},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cgroup":{"cpuacct":{"total":{"ns":3528524654}},"memory":{"mem":{"usage":{"bytes":139264}}}},"cpu":{"system":{"ticks":290},"total":{"ticks":870,"time":{"ms":10},"value":0},"user":{"ticks":580,"time":{"ms":10}}},"handles":{"limit":{"hard":1048576,"soft":1024},"open":12},"info":{"ephemeral_id":"14c8ce41-c547-46b8-8cf1-e6178435730b","uptime":{"ms":123086},"version":"8.2.0"},"memstats":{"gc_next":56125168,"memory_alloc":34466024,"memory_total":146740832,"rss":153948160},"runtime":{"goroutines":34}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":1}},"output":{"events":{"active":0}},"pipeline":{"clients":1,"events":{"active":4117}}},"registrar":{"states":{"current":0}},"system":{"load":{"1":0.02,"15":0.12,"5":0.15,"norm":{"1":0.0033,"15":0.02,"5":0.025}}}},"ecs.version":"1.6.0"}}
{"log.level":"error","@timestamp":"2022-05-03T09:54:31.690-0700","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":150},"message":"Failed to connect to backoff(elasticsearch(https://localhost:9200)): Connection marked as failed because the onConnect callback failed: Elasticsearch is too old. Please upgrade the instance. If you would like to connect to older instances set output.elasticsearch.allow_older_versions to true. ES=8.1.3, Beat=8.2.0.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:54:31.690-0700","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":141},"message":"Attempting to reconnect to backoff(elasticsearch(https://localhost:9200)) with 7 reconnect attempt(s)","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:54:31.692-0700","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":287},"message":"Attempting to connect to Elasticsearch version 8.1.3","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:54:59.229-0700","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":184},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cgroup":{"cpuacct":{"total":{"ns":3471328721}},"memory":{"mem":{"usage":{"bytes":-122880}}}},"cpu":{"system":{"ticks":290},"total":{"ticks":900,"time":{"ms":30},"value":0},"user":{"ticks":610,"time":{"ms":30}}},"handles":{"limit":{"hard":1048576,"soft":1024},"open":12},"info":{"ephemeral_id":"14c8ce41-c547-46b8-8cf1-e6178435730b","uptime":{"ms":153088},"version":"8.2.0"},"memstats":{"gc_next":58292848,"memory_alloc":29328592,"memory_total":147061912,"rss":153948160},"runtime":{"goroutines":34}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":1}},"output":{"events":{"active":0},"read":{"bytes":1109},"write":{"bytes":614}},"pipeline":{"clients":1,"events":{"active":4117,"retry":501}}},"registrar":{"states":{"current":0}},"system":{"load":{"1":0.01,"15":0.12,"5":0.13,"norm":{"1":0.0017,"15":0.02,"5":0.0217}}}},"ecs.version":"1.6.0"}}
{"log.level":"error","@timestamp":"2022-05-03T09:55:04.413-0700","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":150},"message":"Failed to connect to backoff(elasticsearch(https://localhost:9200)): Connection marked as failed because the onConnect callback failed: Elasticsearch is too old. Please upgrade the instance. If you would like to connect to older instances set output.elasticsearch.allow_older_versions to true. ES=8.1.3, Beat=8.2.0.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:55:04.414-0700","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":141},"message":"Attempting to reconnect to backoff(elasticsearch(https://localhost:9200)) with 8 reconnect attempt(s)","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:55:04.414-0700","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":287},"message":"Attempting to connect to Elasticsearch version 8.1.3","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:55:29.226-0700","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":184},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cgroup":{"cpuacct":{"total":{"ns":3315618088}},"memory":{"mem":{"usage":{"bytes":-1007616}}}},"cpu":{"system":{"ticks":310,"time":{"ms":20}},"total":{"ticks":940,"time":{"ms":40},"value":0},"user":{"ticks":630,"time":{"ms":20}}},"handles":{"limit":{"hard":1048576,"soft":1024},"open":12},"info":{"ephemeral_id":"14c8ce41-c547-46b8-8cf1-e6178435730b","uptime":{"ms":183086},"version":"8.2.0"},"memstats":{"gc_next":58292848,"memory_alloc":29490904,"memory_total":147224224,"rss":153059328},"runtime":{"goroutines":34}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":1}},"output":{"events":{"active":0},"read":{"bytes":1109},"write":{"bytes":614}},"pipeline":{"clients":1,"events":{"active":4117,"retry":501}}},"registrar":{"states":{"current":0}},"system":{"load":{"1":0,"15":0.11,"5":0.12,"norm":{"1":0,"15":0.0183,"5":0.02}}}},"ecs.version":"1.6.0"}}
{"log.level":"error","@timestamp":"2022-05-03T09:55:49.678-0700","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":150},"message":"Failed to connect to backoff(elasticsearch(https://localhost:9200)): Connection marked as failed because the onConnect callback failed: Elasticsearch is too old. Please upgrade the instance. If you would like to connect to older instances set output.elasticsearch.allow_older_versions to true. ES=8.1.3, Beat=8.2.0.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:55:49.678-0700","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":141},"message":"Attempting to reconnect to backoff(elasticsearch(https://localhost:9200)) with 9 reconnect attempt(s)","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:55:49.679-0700","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":287},"message":"Attempting to connect to Elasticsearch version 8.1.3","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-03T09:55:59.226-0700","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":184},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cgroup":{"cpuacct":{"total":{"ns":3117305294}},"memory":{"mem":{"usage":{"bytes":32768}}}},"cpu":{"system":{"ticks":320,"time":{"ms":10}},"total":{"ticks":960,"time":{"ms":20},"value":0},"user":{"ticks":640,"time":{"ms":10}}},"handles":{"limit":{"hard":1048576,"soft":1024},"open":12},"info":{"ephemeral_id":"14c8ce41-c547-46b8-8cf1-e6178435730b","uptime":{"ms":213086},"version":"8.2.0"},"memstats":{"gc_next":58292848,"memory_alloc":30027680,"memory_sys":262144,"memory_total":147761000,"rss":153059328},"runtime":{"goroutines":34}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":1}},"output":{"events":{"active":0},"read":{"bytes":1109},"write":{"bytes":614}},"pipeline":{"clients":1,"events":{"active":4117,"retry":501}}},"registrar":{"states":{"current":0}},"system":{"load":{"1":0,"15":0.11,"5":0.1,"norm":{"1":0,"15":0.0183,"5":0.0167}}}},"ecs.version":"1.6.0"}}



Good the filebeat module started... and should be listening...

BAD... filebeat can not connect to elastic... what version of elastic are you using? easier to probably use the same filebeat version as elastic...

You should probably match it with the filebeat version.

"message":"Failed to connect to backoff(elasticsearch(https://localhost:9200)): Connection marked as failed because the onConnect callback failed: Elasticsearch is too old. Please upgrade the instance. If you would like to connect to older instances set output.elasticsearch.allow_older_versions to true. ES=8.1.3, 

I have not tried the
output.elasticsearch.allow_older_versions: true
So I can not comment on how that works

Running Elasticsearch 8.1.3. I ran apt update && apt upgrade but it only upgraded Logstash. I can seem to figure out how to update/upgrade Elasticsearch

Why not just download filebeat 8.1.3 and try.

I think 8.2 Elasticsearch is just rolling out today... perhaps in a couple hours it will be available..

Or try the setting... in the filebeat.yml

output.elasticsearch.allow_older_versions: true

Its working! I installed Filebeat 8.1.3

Thank you for all you help Stephen! :slight_smile:

1 Like