Where is my data of fortinet?

Elastic Stack Elasticsearch
1

Hello, I installed Elasticsearch and kibana and filebeat in ubuntu 22.04 VM and i installed FortiGate 7.2.0 in other VM in VMware workstation, I follow the steps to upload the Fortinet logs in elastic and kibana as the first screenshot, and the data is successfully received from the Filebeat Fortinet module but when i clic in "security App" i don't find anything

image
image1299×607 37.3 KB

image
image1355×601 132 KB

this my nano /etc/filebeat/modules.d/fortinet.yml:

- module: fortinet
  firewall:
    enabled: true

    # Set which input to use between tcp, udp (default) or file.
    var.input: udp

    # The interface to listen to syslog traffic. Defaults to
    # localhost. Set to 0.0.0.0 to bind to all available interfaces.
    var.syslog_host: 192.168.37.2

    # The port to listen for syslog traffic. Defaults to 9004.
    var.syslog_port: 9004

    # Set internal interfaces. used to override parsed network.direction
    # based on a tagged interface. Both internal and external interfaces must be
    # set to leverage this functionality.

and this my config log syslogd setting of fortigate :

FortiGate-VM64 # config log syslogd setting 

FortiGate-VM64 (setting) # show 
config log syslogd setting
    set status enable
    set server "192.168.37.2"
    set port 9004
end

FortiGate-VM64 (setting) #
2

@morad_della3 Welcome to the community! A community of volunteers.
Please be patient on waiting for a response.

Ok you need to provide more data..

What version of the Elastic Stack are you using?

How did you install filebeat? (.rpm, deb?)?

Did you run the setup command (this is crucial)
sudo filbeat setup

Please provide your filebeat.yml?

Please Go Into Kibana -> Dev Tools and run the command and provide the results.

GET _cat/indices?v

Also please provide the filebeat startup logs (assuming you installed as a package)
Stop then Start filebeat and capture the logs

journalctl -u filebeat.service

Did you try as suggested
var.syslog_host: 0.0.0.0

Have you tested connectivity between the fortinet host and the filebeat host? are they on the same host or different hosts?

Are you sending the forinet over udp or tcp (I did not see that defined)

Also from the docs here so this module as not been tested againd Fortinet 7.2

Compatibility

This module has been tested against FortiOS version 6.0.x and 6.2.x. Versions above this are expected to work but have not been tested.

3

@stephenb thanks for your response and sorry for my hurry because i have an end of study project and the time will finish soon
the version of elk is : 8.3.2
How did you install filebeat?deb
Did you run the setup command (this is crucial) yes
the result:



[spoiler]elkfiras[/spoiler]@[spoiler]elkfiras[/spoiler]:~$ sudo filebeat setup
Index setup finished.
Loading dashboards (Kibana must be running and reachable)
Loaded dashboards
Loaded Ingest pipelines

+Please provide your filebeat.yml?

# ============================== Filebeat inputs ===============================

filebeat.inputs:

# Each - is an input. Most options can be set at the input level, so
# you can use different inputs for various configurations.
# Below are the input specific configurations.

# filestream is an input for collecting log messages from files.
- type: filestream

  # Unique ID among all inputs, an ID is required.
  id: my-filestream-id

  # Change to true to enable this input configuration.
  enabled: true

  # Paths that should be crawled and fetched. Glob based paths.
  paths:
    - /var/log/*.log
# ============================== Filebeat modules ==============================

filebeat.config.modules:
  # Glob pattern for configuration loading
  path: ${path.config}/modules.d/*.yml

  # Set to true to enable config reloading
  reload.enabled: true

  # Period on which files under path should be checked for changes
  #reload.period: 10s
# ======================= Elasticsearch template setting =======================

setup.template.settings:
  index.number_of_shards: 1
  #index.codec: best_compression
  #_source.enabled: false
setup.kibana:

  # Kibana Host
  # Scheme and port can be left out and will be set to the default (http and 56>
  # In case you specify and additional path, the scheme is required: http://loc>
  # IPv6 addresses should always be defined as: https://[2001:db8::1]:5601
  host: "https://192.168.37.2:5601"
  ssl.verification_mode: none
  # Kibana Space ID
  # ID of the Kibana Space into which the dashboards should be loaded. By defau>
  # the Default Space will be used.
  #space.id:
# ---------------------------- Elasticsearch Output -------------------->
output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["localhost:9200"]

  # Protocol - either `http` (default) or `https`.
  protocol: "https"

  # Authentication credentials - either API key or username/password.
  #api_key: "id:api_key"
  username: "elastic"
  password: "passok"
  ssl:
    enabled: true
    ca_trusted_fingerprint: "58809f135a319b6e02d22882bc4f5df3c5f289f55ca>
# ================================= Processors =========================>
processors:
  - add_host_metadata:
      when.not.contains.tags: forwarded
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - add_kubernetes_metadata: ~

# ================================= Migration ==========================>

# This allows to enable 6.7 migration aliases
#migration.6_to_7.enabled: true
setup.ilm.overwrite: true

Please Go Into Kibana -> Dev Tools and run the command and provide the results
this the result :

health status index                                     uuid                   pri rep docs.count docs.deleted store.size pri.store.size
yellow open   .items-default-000001                     l4ffGlFIRKyvo5cBA6uKew   1   1          0            0       225b           225b
yellow open   .lists-default-000001                     KVeHeuGNQ6-kT8jxjaFfpw   1   1          0            0       225b           225b
yellow open   .ds-filebeat-8.3.2-2022.07.25-000001      7rKdMiWkRC2DLKQ66GlAew   1   1      16750            0      7.9mb          7.9mb
green  open   metrics-endpoint.metadata_current_default LXlviM3mTii7cUSdnsACmw   1   0          0            0       225b           225

b

 sudo systemctl stop filebeat.service
elkfiras@elkfiras:~$ sudo systemctl start filebeat.service
[spoiler]elkfiras ellkfiras~$ journalctl -u filebeat.service
juil. 25 12:25:56 elkfiras systemd[1]: Started Filebeat sends log files >
juil. 25 12:25:56 elkfiras filebeat[9849]: {"log.level":"info","@timesta>
juil. 25 12:25:56 elkfiras filebeat[9849]: {"log.level":"info","@timesta>
juil. 25 12:25:59 elkfiras filebeat[9849]: {"log.level":"warn","@timesta>
juil. 25 12:25:59 elkfiras filebeat[9849]: {"log.level":"info","@timesta>
juil. 25 12:25:59 elkfiras filebeat[9849]: {"log.level":"info","@timesta>
juil. 25 12:25:59 elkfiras filebeat[9849]: {"log.level":"info","@timesta>
juil. 25 12:25:59 elkfiras filebeat[9849]: {"log.level":"info","@timesta>
juil. 25 12:25:59 elkfiras filebeat[9849]: {"log.level":"info","@timesta>
juil. 25 12:25:59 elkfiras filebeat[9849]: {"log.level":"info","@timesta>
juil. 25 12:25:59 elkfiras filebeat[9849]: {"log.level":"info","@timesta>
juil. 25 12:25:59 elkfiras filebeat[9849]: {"log.level":"warn","@timesta>
juil. 25 12:25:59 elkfiras filebeat[9849]: {"log.level":"info","@timesta>
juil. 25 12:25:59  elkfiras filebeat[9849]: {"log.level":"info","@timesta>
juil. 25 12:25:59elkfiras filebeat[9849]: {"log.level":"info","@timesta>
juil. 25 12:25:59 elkfirasfilebeat[9849]: {"log.level":"info","@timesta>
juil. 25 12:25:59elkfirasfilebeat[9849]: {"log.level":"info","@timesta>
juil. 25 12:25:59 elkfiras filebeat[9849]: {"log.level":"info","@timesta>
juil. 25 12:25:59 elkfiras filebeat[9849]: {"log.level":"info","@timesta>
juil. 25 12:25:59 elkfiras filebeat[9849]: {"log.level":"info","@timesta>
juil. 25 12:25:59elkfiras filebeat[9849]: {"log.level":"info","@timesta>
juil. 25 12:25:59 elkfiras filebeat[9849]: {"log.level":"info","@timesta>
juil. 25 12:25:59 filebeat[9849]: {"log.level":"info","@timesta>

+Did you try as suggested
`var.syslog_host: 0.0.0.0` ?yes
+Have you tested connectivity between the fortinet host and the filebeat host? yes 
 are they on the same host or different hosts? there is different host 
+Are you sending the forinet over udp or tcp (I did not see that defined) udp 
+Also from the docs here so this module as not been tested againd Fortinet 7.2 : that's mean it will not work with this version?
4

Understood, but your timelines are no more important than anyone else.

Why do you have this enabled? This is just complicating the debug

- type: filestream

  # Unique ID among all inputs, an ID is required.
  id: my-filestream-id

  # Change to true to enable this input configuration.
  enabled: true

You don't need that turn it off for now
enabled: false

Then we will clean up and start over.

Stop filebeat

Then go to Kibana and Delete the filebeat-8.3.2 datastream

Then Run setup again

Then start filebeat

Then this again.
GET _cat/indices?v

Then run and provide a couple samples

GET filebeat-8.3.2/_search

Also you truncated the log lines so they are of no use, we need to see the whole logs lines otherwise they are useless.

journalctl -u filebeat.service

How did you test connectivity between the two hosts?

Did you try to to use tcp so you could test the connectivity with telnet?

5

You don't need that turn it off for now
enabled: false i did

Then we will clean up and start over.

Stop filebeat i did

Then go to Kibana and Delete the filebeat-8.3.2 datastream i did

Then Run setup again "i did"

Then start filebeat "i did"

Then this again.

`GET _cat/indices?v`
health status index                                     uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   metrics-endpoint.metadata_current_default LXlviM3mTii7cUSdnsACmw   1   0          0            0       225b           225b
yellow open   .ds-filebeat-8.3.2-2022.07.30-000001      I6gjowMmRRqcl3qb92PEyA   1   1          0            0       225b           225b
yellow open   .lists-default-000001                     KVeHeuGNQ6-kT8jxjaFfpw   1   1          0            0       225b           225b
yellow open   .items-default-000001                     l4ffGlFIRKyvo5cBA6uKew   1   1          0            0       225b           225b

Then run and provide a couple samples

GET filebeat-8.3.2/_search

  "took": 2,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 0,
      "relation": "eq"
    },
    "max_score": null,
    "hits": []
  }
}
6

So that means you're not ingesting any fortinet data into elasticsearch and you need to look at the filebeat logs in detail to see why it's failing.

Usually it's failing for one or two reasons

One it's not actually reading the fortinet and/or the fortinet is actually forwarding the logs to the correct host and port.

Or two? It's reading the fortinet logs, but when it tries to write them, there is a failure in the writing That would be in the filebeat logs.

You can just start filebeat from the command line and watch it run...

filebeat -e

But you need those logs, not truncated

7

sudo filebeat -e

{"log.level":"info","@timestamp":"2022-07-31T10:22:56.663Z","log.origin":{"file.name":"instance/beat.go","file.line":702},"message":"Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T10:22:56.664Z","log.origin":{"file.name":"instance/beat.go","file.line":710},"message":"Beat ID: 18f18c9b-6051-49ef-a4ce-e9cce3299b83","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2022-07-31T10:22:59.670Z","log.logger":"add_cloud_metadata","log.origin":{"file.name":"add_cloud_metadata/provider_aws_ec2.go","file.line":81},"message":"read token request for getting IMDSv2 token returns empty: Put \"http://169.254.169.254/latest/api/token\": context deadline exceeded (Client.Timeout exceeded while awaiting headers). No token in the metadata request will be used.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T10:22:59.672Z","log.origin":{"file.name":"instance/beat.go","file.line":391},"message":"filebeat stopped.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2022-07-31T10:22:59.672Z","log.origin":{"file.name":"instance/beat.go","file.line":1051},"message":"Exiting: data path already locked by another beat. Please make sure that multiple beats are not sharing the same data path (path.data).","service.name":"filebeat","ecs.version":"1.6.0"}
Exiting: data path already locked by another beat. Please make sure that multiple beats are not sharing the same data path (path.data).
8

do you mean this cmd : sudo filebeat setup -e ???

9

You have to stop the filebeat service first 2 can not run at the same time...

systemctl stop filebeat

filebeat -e
10 
root@elkfiras:/home/elkfiras# filebeat -e
{"log.level":"info","@timestamp":"2022-07-31T17:20:16.558Z","log.origin":{"file.name":"instance/beat.go","file.line":702},"message":"Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:20:16.559Z","log.origin":{"file.name":"instance/beat.go","file.line":710},"message":"Beat ID: 18f18c9b-6051-49ef-a4ce-e9cce3299b83","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2022-07-31T17:20:19.565Z","log.logger":"add_cloud_metadata","log.origin":{"file.name":"add_cloud_metadata/provider_aws_ec2.go","file.line":81},"message":"read token request for getting IMDSv2 token returns empty: Put \"http://169.254.169.254/latest/api/token\": context deadline exceeded (Client.Timeout exceeded while awaiting headers). No token in the metadata request will be used.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:20:19.574Z","log.logger":"seccomp","log.origin":{"file.name":"seccomp/seccomp.go","file.line":124},"message":"Syscall filter successfully installed","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:20:19.576Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1076},"message":"Beat info","service.name":"filebeat","system_info":{"beat":{"path":{"config":"/etc/filebeat","data":"/var/lib/filebeat","home":"/usr/share/filebeat","logs":"/var/log/filebeat"},"type":"filebeat","uuid":"18f18c9b-6051-49ef-a4ce-e9cce3299b83"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-07-31T17:20:19.576Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1085},"message":"Build info","service.name":"filebeat","system_info":{"build":{"commit":"45f722f492dcf1d13698c6cf618b339b1d4907be","libbeat":"8.3.2","time":"2022-07-06T10:12:50.000Z","version":"8.3.2"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-07-31T17:20:19.576Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1088},"message":"Go runtime info","service.name":"filebeat","system_info":{"go":{"os":"linux","arch":"amd64","max_procs":2,"version":"go1.18.2"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-07-31T17:20:19.584Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1092},"message":"Host info","service.name":"filebeat","system_info":{"host":{"architecture":"x86_64","boot_time":"2022-07-31T00:55:46Z","containerized":false,"name":"elkfiras","ip":["127.0.0.1/8","::1/128","192.168.37.2/24","fe80::20c:29ff:fe65:82ff/64"],"kernel_version":"5.15.0-41-generic","mac":["00:0c:29:65:82:ff"],"os":{"type":"linux","family":"debian","platform":"ubuntu","name":"Ubuntu","version":"22.04 (Jammy Jellyfish)","major":22,"minor":4,"patch":0,"codename":"jammy"},"timezone":"UTC","timezone_offset_sec":0,"id":"24ec3a89c2b5417b9c2a0e9755bb64bc"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-07-31T17:20:19.590Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1121},"message":"Process info","service.name":"filebeat","system_info":{"process":{"capabilities":{"inheritable":null,"permitted":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read","38","39","40"],"effective":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read","38","39","40"],"bounding":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read","38","39","40"],"ambient":null},"cwd":"/home/elkfiras","exe":"/usr/share/filebeat/bin/filebeat","name":"filebeat","pid":12885,"ppid":12876,"seccomp":{"mode":"filter","no_new_privs":true},"start_time":"2022-07-31T17:20:16.040Z"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-07-31T17:20:19.591Z","log.origin":{"file.name":"instance/beat.go","file.line":293},"message":"Setup Beat: filebeat; Version: 8.3.2","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2022-07-31T17:20:19.609Z","log.logger":"cfgwarn","log.origin":{"file.name":"tlscommon/config.go","file.line":102},"message":"DEPRECATED: Treating the CommonName field on X.509 certificates as a host name when no Subject Alternative Names are present is going to be removed. Please update your certificates if needed. Will be removed in version: 8.0.0","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:20:19.610Z","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":108},"message":"elasticsearch url: https://localhost:9200","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:20:19.611Z","log.logger":"publisher","log.origin":{"file.name":"pipeline/module.go","file.line":113},"message":"Beat name: elkfiras","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:20:19.611Z","log.logger":"modules","log.origin":{"file.name":"fileset/modules.go","file.line":108},"message":"Enabled modules/filesets: ","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:20:19.613Z","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":143},"message":"Starting metrics logging every 30s","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:20:19.615Z","log.origin":{"file.name":"instance/beat.go","file.line":470},"message":"filebeat start running.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:20:19.651Z","log.origin":{"file.name":"memlog/store.go","file.line":134},"message":"Finished loading transaction log file for '/var/lib/filebeat/registry/filebeat'. Active transaction id=1373","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:20:19.653Z","log.logger":"registrar","log.origin":{"file.name":"registrar/registrar.go","file.line":109},"message":"States Loaded from registrar: 0","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:20:19.655Z","log.logger":"crawler","log.origin":{"file.name":"beater/crawler.go","file.line":71},"message":"Loading Inputs: 1","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:20:19.655Z","log.logger":"crawler","log.origin":{"file.name":"beater/crawler.go","file.line":117},"message":"starting input, keys present on the config: [filebeat.inputs.0.enabled filebeat.inputs.0.id filebeat.inputs.0.paths.0 filebeat.inputs.0.type]","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:20:19.658Z","log.logger":"crawler","log.origin":{"file.name":"beater/crawler.go","file.line":148},"message":"Starting input (ID: 11337388005444501392)","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:20:19.659Z","log.logger":"crawler","log.origin":{"file.name":"beater/crawler.go","file.line":106},"message":"Loading and starting Inputs completed. Enabled inputs: 1","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:20:19.659Z","log.logger":"input.filestream","log.origin":{"file.name":"compat/compat.go","file.line":111},"message":"Input filestream starting","service.name":"filebeat","id":"my-filestream-id","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:20:19.659Z","log.origin":{"file.name":"cfgfile/reload.go","file.line":164},"message":"Config reloader started","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:20:22.568Z","log.logger":"add_cloud_metadata","log.origin":{"file.name":"add_cloud_metadata/add_cloud_metadata.go","file.line":102},"message":"add_cloud_metadata: hosting provider type not detected.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:20:23.571Z","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":139},"message":"Connecting to backoff(elasticsearch(https://localhost:9200))","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:20:23.642Z","log.logger":"tls","log.origin":{"file.name":"tlscommon/tls_config.go","file.line":162},"message":"'ca_trusted_fingerprint' set, looking for matching fingerprints","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:20:23.643Z","log.logger":"tls","log.origin":{"file.name":"tlscommon/tls_config.go","file.line":173},"message":"CA certificate matching 'ca_trusted_fingerprint' found, adding it to 'certificate_authorities'","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:20:23.659Z","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":291},"message":"Attempting to connect to Elasticsearch version 8.3.2","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:20:23.670Z","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":291},"message":"Attempting to connect to Elasticsearch version 8.3.2","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:20:23.671Z","log.logger":"index-management","log.origin":{"file.name":"idxmgmt/std.go","file.line":231},"message":"Auto ILM enable success.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:20:23.680Z","log.logger":"index-management.ilm","log.origin":{"file.name":"ilm/std.go","file.line":128},"message":"ILM policy filebeat successfully created.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:20:23.680Z","log.logger":"index-management","log.origin":{"file.name":"idxmgmt/std.go","file.line":366},"message":"Set settings.index.lifecycle.name in template to {filebeat {\"policy\":{\"phases\":{\"hot\":{\"actions\":{\"rollover\":{\"max_age\":\"30d\",\"max_size\":\"50gb\"}}}}}}} as ILM is enabled.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:20:23.720Z","log.logger":"template","log.origin":{"file.name":"template/load.go","file.line":245},"message":"Existing template will be overwritten, as overwrite is enabled.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:20:25.930Z","log.logger":"template_loader","log.origin":{"file.name":"template/load.go","file.line":159},"message":"Try loading template filebeat-8.3.2 to Elasticsearch","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:20:26.098Z","log.logger":"template_loader","log.origin":{"file.name":"template/load.go","file.line":127},"message":"Template with name \"filebeat-8.3.2\" loaded.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:20:26.102Z","log.logger":"template_loader","log.origin":{"file.name":"template/load.go","file.line":143},"message":"Data stream with name \"filebeat-8.3.2\" already exists.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:20:26.102Z","log.logger":"index-management","log.origin":{"file.name":"idxmgmt/std.go","file.line":267},"message":"Loaded index template.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:20:26.105Z","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":147},"message":"Connection to backoff(elasticsearch(https://localhost:9200)) established","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2022-07-31T17:20:29.662Z","log.origin":{"file.name":"cfgfile/reload.go","file.line":273},"message":"Error loading config from file '/etc/filebeat/modules.d/fortinet.yml', error invalid config: yaml: line 3: did not find expected key","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:20:49.623Z","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":185},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cgroup":{"cpu":{"id":"session-33.scope"},"memory":{"id":"session-33.scope","mem":{"usage":{"bytes":138010624}}}},"cpu":{"system":{"ticks":460,"time":{"ms":460}},"total":{"ticks":3060,"time":{"ms":3060},"value":0},"user":{"ticks":2600,"time":{"ms":2600}}},"handles":{"limit":{"hard":1048576,"soft":1024},"open":36},"info":{"ephemeral_id":"679a32c7-b926-45db-a8e2-bd4255ae7d34","name":"filebeat","uptime":{"ms":33174},"version":"8.3.2"},"memstats":{"gc_next":29926312,"memory_alloc":14720128,"memory_sys":109563928,"memory_total":320591248,"rss":121556992},"runtime":{"goroutines":103}},"filebeat":{"events":{"added":5,"done":5},"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0},"reloads":1,"scans":2},"output":{"events":{"acked":5,"active":0,"batches":1,"total":5},"read":{"bytes":4320},"type":"elasticsearch","write":{"bytes":378403}},"pipeline":{"clients":25,"events":{"active":0,"published":5,"retry":5,"total":5},"queue":{"acked":5,"max_events":4096}}},"registrar":{"states":{"current":0}},"system":{"cpu":{"cores":2},"load":{"1":0.39,"15":0.25,"5":0.23,"norm":{"1":0.195,"15":0.125,"5":0.115}}}},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-07-31T17:21:19.623Z","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":185},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cgroup":{"memory":{"mem":{"usage":{"bytes":61440}}}},"cpu":{"system":{"ticks":470,"time":{"ms":10}},"total":{"ticks":3090,"time":{"ms":30},"value":0},"user":{"ticks":2620,"time":{"ms":20}}},"handles":{"limit":{"hard":1048576,"soft":1024},"open":36},"info":{"ephemeral_id":"679a32c7-b926-45db-a8e2-bd4255ae7d34","uptime":{"ms":63177},"version":"8.3.2"},"memstats":{"gc_next":29926312,"memory_alloc":15128544,"memory_total":320999664,"rss":121556992},"runtime":{"goroutines":103}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0},"scans":3},"output":{"events":{"active":0}},"pipeline":{"clients":25,"events":{"active":0}}},"registrar":{"states":{"current":0}},"system":{"load":{"1":0.43,"15":0.26,"5":0.26,"norm":{"1":0.215,"15":0.13,"5":0.13}}}},"ecs.version":"1.6.0"}}
^C{"log.level":"info","@timestamp":"2022-07-31T17:21:29.385Z","log.origin":{"file.name":"beater/filebeat.go","file.line":425},"message":"Stopping filebeat","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:21:29.386Z","log.origin":{"file.name":"beater/crawler.go","file.line":155},"message":"Stopping Crawler","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:21:29.386Z","log.origin":{"file.name":"beater/crawler.go","file.line":165},"message":"Stopping 1 inputs","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:21:29.386Z","log.origin":{"file.name":"cfgfile/reload.go","file.line":190},"message":"Dynamic config reloader stopped","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:21:29.386Z","log.logger":"crawler","log.origin":{"file.name":"beater/crawler.go","file.line":170},"message":"Stopping input: 11337388005444501392","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:21:29.387Z","log.logger":"input.filestream","log.origin":{"file.name":"compat/compat.go","file.line":132},"message":"Input 'filestream' stopped","service.name":"filebeat","id":"my-filestream-id","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:21:29.387Z","log.origin":{"file.name":"beater/crawler.go","file.line":185},"message":"Crawler stopped","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:21:29.387Z","log.logger":"registrar","log.origin":{"file.name":"registrar/registrar.go","file.line":132},"message":"Stopping Registrar","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:21:29.387Z","log.logger":"registrar","log.origin":{"file.name":"registrar/registrar.go","file.line":166},"message":"Ending Registrar","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:21:29.388Z","log.logger":"registrar","log.origin":{"file.name":"registrar/registrar.go","file.line":137},"message":"Registrar stopped","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:21:29.388Z","log.logger":"input.filestream","log.origin":{"file.name":"filestream/input.go","file.line":321},"message":"Reader was closed. Closing.","service.name":"filebeat","id":"my-filestream-id","source_file":"filestream::my-filestream-id::native::795008-64768","path":"/var/log/vmware-network.4.log","state-id":"native::795008-64768","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:21:29.391Z","log.logger":"input.filestream","log.origin":{"file.name":"filestream/input.go","file.line":321},"message":"Reader was closed. Closing.","service.name":"filebeat","id":"my-filestream-id","source_file":"filestream::my-filestream-id::native::810340-64768","path":"/var/log/vmware-network.2.log","state-id":"native::810340-64768","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:21:29.392Z","log.logger":"input.filestream","log.origin":{"file.name":"filestream/input.go","file.line":321},"message":"Reader was closed. Closing.","service.name":"filebeat","id":"my-filestream-id","source_file":"filestream::my-filestream-id::native::797416-64768","path":"/var/log/alternatives.log","state-id":"native::797416-64768","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:21:29.392Z","log.logger":"input.filestream","log.origin":{"file.name":"filestream/input.go","file.line":321},"message":"Reader was closed. Closing.","service.name":"filebeat","id":"my-filestream-id","source_file":"filestream::my-filestream-id::native::794985-64768","path":"/var/log/vmware-vmsvc-root.log","state-id":"native::794985-64768","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:21:29.392Z","log.logger":"input.filestream","log.origin":{"file.name":"filestream/input.go","file.line":321},"message":"Reader was closed. Closing.","service.name":"filebeat","id":"my-filestream-id","source_file":"filestream::my-filestream-id::native::797397-64768","path":"/var/log/vmware-network.9.log","state-id":"native::797397-64768","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:21:29.394Z","log.logger":"input.filestream","log.origin":{"file.name":"filestream/input.go","file.line":321},"message":"Reader was closed. Closing.","service.name":"filebeat","id":"my-filestream-id","source_file":"filestream::my-filestream-id::native::801552-64768","path":"/var/log/kern.log","state-id":"native::801552-64768","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:21:29.395Z","log.logger":"input.filestream","log.origin":{"file.name":"filestream/input.go","file.line":321},"message":"Reader was closed. Closing.","service.name":"filebeat","id":"my-filestream-id","source_file":"filestream::my-filestream-id::native::795007-64768","path":"/var/log/vmware-network.5.log","state-id":"native::795007-64768","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:21:29.395Z","log.logger":"input.filestream","log.origin":{"file.name":"filestream/input.go","file.line":321},"message":"Reader was closed. Closing.","service.name":"filebeat","id":"my-filestream-id","source_file":"filestream::my-filestream-id::native::797398-64768","path":"/var/log/vmware-network.log","state-id":"native::797398-64768","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:21:29.395Z","log.logger":"input.filestream","log.origin":{"file.name":"filestream/input.go","file.line":321},"message":"Reader was closed. Closing.","service.name":"filebeat","id":"my-filestream-id","source_file":"filestream::my-filestream-id::native::798004-64768","path":"/var/log/bootstrap.log","state-id":"native::798004-64768","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:21:29.396Z","log.logger":"input.filestream","log.origin":{"file.name":"filestream/input.go","file.line":321},"message":"Reader was closed. Closing.","service.name":"filebeat","id":"my-filestream-id","source_file":"filestream::my-filestream-id::native::794967-64768","path":"/var/log/ubuntu-advantage-timer.log","state-id":"native::794967-64768","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:21:29.397Z","log.logger":"input.filestream","log.origin":{"file.name":"filestream/input.go","file.line":321},"message":"Reader was closed. Closing.","service.name":"filebeat","id":"my-filestream-id","source_file":"filestream::my-filestream-id::native::801464-64768","path":"/var/log/dpkg.log","state-id":"native::801464-64768","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:21:29.397Z","log.logger":"input.filestream","log.origin":{"file.name":"filestream/input.go","file.line":321},"message":"Reader was closed. Closing.","service.name":"filebeat","id":"my-filestream-id","source_file":"filestream::my-filestream-id::native::801769-64768","path":"/var/log/fontconfig.log","state-id":"native::801769-64768","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:21:29.398Z","log.logger":"input.filestream","log.origin":{"file.name":"filestream/input.go","file.line":321},"message":"Reader was closed. Closing.","service.name":"filebeat","id":"my-filestream-id","source_file":"filestream::my-filestream-id::native::798058-64768","path":"/var/log/vmware-vmtoolsd-root.log","state-id":"native::798058-64768","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:21:29.398Z","log.logger":"input.filestream","log.origin":{"file.name":"filestream/input.go","file.line":321},"message":"Reader was closed. Closing.","service.name":"filebeat","id":"my-filestream-id","source_file":"filestream::my-filestream-id::native::795221-64768","path":"/var/log/vmware-network.8.log","state-id":"native::795221-64768","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:21:29.399Z","log.logger":"input.filestream","log.origin":{"file.name":"filestream/input.go","file.line":321},"message":"Reader was closed. Closing.","service.name":"filebeat","id":"my-filestream-id","source_file":"filestream::my-filestream-id::native::795295-64768","path":"/var/log/vmware-network.6.log","state-id":"native::795295-64768","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:21:29.399Z","log.logger":"input.filestream","log.origin":{"file.name":"filestream/input.go","file.line":321},"message":"Reader was closed. Closing.","service.name":"filebeat","id":"my-filestream-id","source_file":"filestream::my-filestream-id::native::810346-64768","path":"/var/log/auth.log","state-id":"native::810346-64768","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:21:29.399Z","log.logger":"input.filestream","log.origin":{"file.name":"filestream/input.go","file.line":321},"message":"Reader was closed. Closing.","service.name":"filebeat","id":"my-filestream-id","source_file":"filestream::my-filestream-id::native::796916-64768","path":"/var/log/vmware-network.3.log","state-id":"native::796916-64768","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:21:29.399Z","log.logger":"input.filestream","log.origin":{"file.name":"filestream/input.go","file.line":321},"message":"Reader was closed. Closing.","service.name":"filebeat","id":"my-filestream-id","source_file":"filestream::my-filestream-id::native::798009-64768","path":"/var/log/ubuntu-advantage.log","state-id":"native::798009-64768","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:21:29.399Z","log.logger":"input.filestream","log.origin":{"file.name":"filestream/input.go","file.line":321},"message":"Reader was closed. Closing.","service.name":"filebeat","id":"my-filestream-id","source_file":"filestream::my-filestream-id::native::798116-64768","path":"/var/log/cloud-init.log","state-id":"native::798116-64768","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:21:29.399Z","log.logger":"input.filestream","log.origin":{"file.name":"filestream/input.go","file.line":321},"message":"Reader was closed. Closing.","service.name":"filebeat","id":"my-filestream-id","source_file":"filestream::my-filestream-id::native::794992-64768","path":"/var/log/vmware-vmsvc-root.1.log","state-id":"native::794992-64768","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:21:29.399Z","log.logger":"input.filestream","log.origin":{"file.name":"filestream/input.go","file.line":321},"message":"Reader was closed. Closing.","service.name":"filebeat","id":"my-filestream-id","source_file":"filestream::my-filestream-id::native::794994-64768","path":"/var/log/vmware-vmsvc-root.2.log","state-id":"native::794994-64768","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:21:29.399Z","log.logger":"input.filestream","log.origin":{"file.name":"filestream/input.go","file.line":321},"message":"Reader was closed. Closing.","service.name":"filebeat","id":"my-filestream-id","source_file":"filestream::my-filestream-id::native::798124-64768","path":"/var/log/cloud-init-output.log","state-id":"native::798124-64768","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:21:29.399Z","log.logger":"input.filestream","log.origin":{"file.name":"filestream/input.go","file.line":321},"message":"Reader was closed. Closing.","service.name":"filebeat","id":"my-filestream-id","source_file":"filestream::my-filestream-id::native::796624-64768","path":"/var/log/vmware-network.7.log","state-id":"native::796624-64768","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:21:29.399Z","log.logger":"input.filestream","log.origin":{"file.name":"filestream/input.go","file.line":321},"message":"Reader was closed. Closing.","service.name":"filebeat","id":"my-filestream-id","source_file":"filestream::my-filestream-id::native::796331-64768","path":"/var/log/vmware-network.1.log","state-id":"native::796331-64768","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:21:29.408Z","log.logger":"input.filestream","log.origin":{"file.name":"filestream/input.go","file.line":321},"message":"Reader was closed. Closing.","service.name":"filebeat","id":"my-filestream-id","source_file":"filestream::my-filestream-id::native::794892-64768","path":"/var/log/vmware-vmsvc-root.3.log","state-id":"native::794892-64768","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:21:29.409Z","log.logger":"input.filestream","log.origin":{"file.name":"compat/compat.go","file.line":124},"message":"Input 'filestream' stopped","service.name":"filebeat","id":"my-filestream-id","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:21:29.424Z","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":193},"message":"Total metrics","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cgroup":{"cpu":{"id":"session-33.scope","stats":{"periods":0,"throttled":{"ns":0,"periods":0}}},"memory":{"id":"session-33.scope","mem":{"usage":{"bytes":138215424}}}},"cpu":{"system":{"ticks":500,"time":{"ms":500}},"total":{"ticks":3130,"time":{"ms":3130},"value":0},"user":{"ticks":2630,"time":{"ms":2630}}},"handles":{"limit":{"hard":1048576,"soft":1024},"open":9},"info":{"ephemeral_id":"679a32c7-b926-45db-a8e2-bd4255ae7d34","name":"filebeat","uptime":{"ms":72976},"version":"8.3.2"},"memstats":{"gc_next":29926312,"memory_alloc":15570696,"memory_sys":109563928,"memory_total":321441816,"rss":121556992},"runtime":{"goroutines":14}},"filebeat":{"events":{"active":0,"added":5,"done":5},"harvester":{"closed":0,"open_files":0,"running":0,"skipped":0,"started":0},"input":{"log":{"files":{"renamed":0,"truncated":0}},"netflow":{"flows":0,"packets":{"dropped":0,"received":0}}}},"libbeat":{"config":{"module":{"running":0,"starts":0,"stops":0},"reloads":1,"scans":6},"output":{"events":{"acked":5,"active":0,"batches":1,"dropped":0,"duplicates":0,"failed":0,"toomany":0,"total":5},"read":{"bytes":4320,"errors":1},"type":"elasticsearch","write":{"bytes":378403,"errors":0}},"pipeline":{"clients":0,"events":{"active":0,"dropped":0,"failed":0,"filtered":0,"published":5,"retry":5,"total":5},"queue":{"acked":5,"max_events":4096}}},"registrar":{"states":{"cleanup":0,"current":0,"update":0},"writes":{"fail":0,"success":0,"total":0}},"system":{"cpu":{"cores":2},"load":{"1":0.74,"15":0.29,"5":0.33,"norm":{"1":0.37,"15":0.145,"5":0.165}}}},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-07-31T17:21:29.425Z","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":194},"message":"Uptime: 1m12.986389323s","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:21:29.425Z","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":161},"message":"Stopping metrics logging.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:21:29.426Z","log.origin":{"file.name":"instance/beat.go","file.line":475},"message":"filebeat stopped.","service.name":"filebeat","ecs.version":"1.6.0"}
root@elkfiras:/home/elkfiras# filebeat -e
{"log.level":"info","@timestamp":"2022-07-31T17:21:38.503Z","log.origin":{"file.name":"instance/beat.go","file.line":702},"message":"Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:21:38.503Z","log.origin":{"file.name":"instance/beat.go","file.line":710},"message":"Beat ID: 18f18c9b-6051-49ef-a4ce-e9cce3299b83","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2022-07-31T17:21:41.510Z","log.logger":"add_cloud_metadata","log.origin":{"file.name":"add_cloud_metadata/provider_aws_ec2.go","file.line":81},"message":"read token request for getting IMDSv2 token returns empty: Put \"http://169.254.169.254/latest/api/token\": context deadline exceeded (Client.Timeout exceeded while awaiting headers). No token in the metadata request will be used.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:21:41.518Z","log.logger":"seccomp","log.origin":{"file.name":"seccomp/seccomp.go","file.line":124},"message":"Syscall filter successfully installed","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-07-31T17:21:41.519Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1076},"message":"Beat info","service.name":"filebeat","system_info":{"beat":{"path":{"config":"/etc/filebeat","data":"/var/lib/filebeat","home":"/usr/share/filebeat","logs":"/var/log/filebeat"},"type":"filebeat","uuid":"18f18c9b-6051-49ef-a4ce-e9cce3299b83"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-07-31T17:21:41.522Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1085},"message":"Build info","service.name":"filebeat","system_info":{"build":{"commit":"45f722f492dcf1d13698c6cf618b339b1d4907be","libbeat":"8.3.2","time":"2022-07-06T10:12:50.000Z","version":"8.3.2"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-07-31T17:21:41.523Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1088},"message":"Go runtime info","service.name":"filebeat","system_info":{"go":{"os":"linux","arch":"amd64","max_procs":2,"version":"go1.18.2"},"ecs.version":"1.6.0"}}
11

@stephenb i see this line from the logs above is an error :
{"log.level":"error","@timestamp":"2022-07-31T17:20:29.662Z","log.origin":{"file.name":"cfgfile/reload.go","file.line":273},"message":"Error loading config from file '/etc/filebeat/modules.d/fortinet.yml', error invalid config: yaml: line 3: did not find expected key","service.name":"filebeat","ecs.version":"1.6.0"}

12

So it looks like you ran more than once...

First one looks like you still had the filestream on... Okay,

Then it looks like you ran it again and only gave me partial logs but that's okay.

That last error is definitely an issue. It basically says the

fortinet.yml

Is malformed / has a syntax error.. That means it's not loading it. That means you're not processing anything

Please paste the entire file fortinet.yml properly formatted and we'll take a look

13 
y# Module: fortinet
# Docs: https://www.elastic.co/guide/en/beats/filebeat/8.3/filebeat-module-fortinet.html

- module: fortinet
  firewall:
    enabled: true

    # Set which input to use between tcp, udp (default) or file.
    var.input: udp

    # The interface to listen to syslog traffic. Defaults to
    # localhost. Set to 0.0.0.0 to bind to all available interfaces.
    var.syslog_host: 192.168.37.2

    # The port to listen for syslog traffic. Defaults to 9004.
    var.syslog_port: 9004

    # Set internal interfaces. used to override parsed network.direction
    # based on a tagged interface. Both internal and external interfaces must be
    # set to leverage this functionality.
    #var.internal_interfaces: [ "LAN" ]

    # Set external interfaces. used to override parsed network.direction
    # based on a tagged interface. Both internal and external interfaces must be
    # set to leverage this functionality.
    #var.external_interfaces: [ "WAN" ]

  clientendpoint:
    enabled: false

    # Set which input to use between udp (default), tcp or file.
    # var.input: udp
    # var.syslog_host: localhost
    # var.syslog_port: 9510

    # Set paths for the log files when file input is used.
    # var.paths:

    # Toggle output of non-ECS fields (default true).
    # var.rsa_fields: true
  clientendpoint:
    enabled: false

    # Set which input to use between udp (default), tcp or file.
    # var.input: udp
    # var.syslog_host: localhost
    # var.syslog_port: 9510

    # Set paths for the log files when file input is used.
    # var.paths:

    # Toggle output of non-ECS fields (default true).
    # var.rsa_fields: true

    # Set custom timezone offset.
    # "local" (default) for system timezone.
    # "+02:00" for GMT+02:00
    # var.tz_offset: local

  fortimail:
    enabled: false

    # Set which input to use between udp (default), tcp or file.
    # var.input: udp
    # var.syslog_host: localhost
    # var.syslog_port: 9529

    # Set paths for the log files when file input is used.
    # var.paths:

    # Toggle output of non-ECS fields (default true).
    # var.rsa_fields: true

    # Set custom timezone offset.
    # "local" (default) for system timezone.
    # "+02:00" for GMT+02:00
    # var.tz_offset: local

  fortimanager:
    enabled: false

    # Set which input to use between udp (default), tcp or file.
    # var.input: udp
    # var.syslog_host: localhost
    # var.syslog_port: 9530

    # Set paths for the log files when file input is used.
    # var.paths:

    # Toggle output of non-ECS fields (default true).
    # var.rsa_fields: true

    # Set custom timezone offset.
    # "local" (default) for system timezone.
    # "+02:00" for GMT+02:00
    # var.tz_offset: local
14

What's that leading y on the first line
y# Module
That should not be there

15

OMG yea right it's an error from me keyboard error

1 Like
16

i delete it

17

Clean it up the start from the command line again..

Look at the logs...

18

i do the "filebeat -e" again?

19

Yes... Then look at the logs... You should see a message about listening on the UDP port

20

i don't see this message , it's many logs but i don't see it there not a messsage of type "error" the most of the messages of type "info"