Filebeat Modules for Syslog-ng files not creating index

I have a Syslog-NG server that has been upgraded from 7.4 to 8.0.1 via YUM on CentOS. I am new to ELK. I have mostly switch syslog data and Palo Alto FW syslog data on the Syslog -NG server. When I set the filebeat config to use the Palo Alto & Syslog modules the index on ES is not created. If I setup the manual input in filebeat for the folder with all my syslog-ng log files the index is created and and a data steam is created. The index is a ds. under hidden indices. The filebeat log debug doesn't tell me much.

## Filebeat.yml ##
  paths:
filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false
setup.template.settings:
  index.number_of_shards: 1
setup.kibana:
  host: "172.16.23.63:5601"
output.elasticsearch:
  hosts: ["172.16.23.61:9200", "172.16.23.62:9200"]
processors:
   - add_host_metadata: ~
logging.level: debug
logging.selectors: ["publish"]
logging.to_files: true
logging.files:
  path: /var/log/filebeat
  name: filebeat-debug
  keepfiles: 7
  permissions: 0644

## panw.yml ## 
# Module: panw
# Docs: https://www.elastic.co/guide/en/beats/filebeat/7.4/filebeat-module-panw.html

- module: panw
  panos:
    enabled: true

    # Set which input to use between syslog (default) or file.
    #var.input:

    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    #var.paths:
#    var.paths: ["/home/log/PAN/STTV-PANMGR.sttv.dea.doj.gov/**/*.log"]


    var.paths:
      - '/home/log/dtx-pan5k-node1.sttv.dea.doj.gov/**/*.log'
      - '/home/log/dtx-pan5k-node2.sttv.dea.doj.gov/**/*.log'
      - '/home/log/sptc-pan5k-node1.sttv.dea.doj.gov/**/*.log'
      - '/home/log/sptc-pan5k-node2.sttv.dea.doj.gov/**/*.log'

    var.input: "file"

## System.yml ## 
# Module: system
# Docs: https://www.elastic.co/guide/en/beats/filebeat/7.4/filebeat-module-system.html

- module: system
  # Syslog
  syslog:
    enabled: true
    #var.paths: ["/home/log/HOSTS/*/*.log"]

    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    #var.paths:

  # Authorization logs
#  auth:
#    enabled: true

    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    #var.paths:

## Setting defined by STVP-OPS ##
    var.paths: ["/home/log/HOSTS/**/*.log"]
    var.paths: ["/var/log/*.log"]

    var.input: "file"


I ended up turning off the PANW module and just using the SYSTEM module with syslog and it started to work.

Also I am still researching why the index is named
.ds-filebeat-8.0.1-2022.03.15-000001 (Data Stream Index).

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.