Need an help for two indices configuration in filebeat for both OS and DB

Elastic Stack Beats
1

Hi Everyone, We are trying to configure filebeat to collect logs for both OS and DB on the server, but we found that OS logs are collecting database logs are not recieving. Can someone Please help on it.

filebeat.inputs:
- type: log
  enabled: true
  ignore_older: 72h
  paths:
     - /var/log/messages
  processors:
     - add_fields:
         fields:
           log_type: operating_system
#------------------------------------------------------------------------for DB
- type: log
  enabled: true
  ignore_older: 72h
  paths:
    - /data/mysqldata/mysqld.log
  processors:
    - add_fields:
        fields:
          log_type: database
#-----------------------------------------------------------------------

close.timeout: 5m
setup.kibana:
  host: "*.*.*.*:5601"
filebeat.config.modules:
  # Glob pattern for configuration loading
  path: ${path.config}/modules.d/*.yml
  # Set to true to enable config reloading
  reload.enabled: true
  # Period on which files under path should be checked for changes
  #reload.period: 10s
setup.template:
    name:    'cli_infra'
    pattern: 'cli_infra*'
    settings:
      index.number_of_shards: 3
      index.number_of_replicas: 1
setup.ilm.enabled: false
setup.template.enabled: false
output.elasticsearch:
    hosts: ["*.*.*.*:9200","*.*.*.*:9200","*.*.*.*:9200"]
    protocol: "https"
    username: '88888888'
    password: '999999999'
    #ssl.truststore.path: /etc/metricbeat/certs/elastic-certificates-http.p12
    ssl.verification_mode: none
    indices:
      - index: 'cli_infra_linux-%{+yyyy.MM.dd}'
        when.equals:
          fields.log_type: 'operating_system'
      - index: 'cli_infra_database-%{[agent.hostname]}-%{+yyyy.MM.dd}'
        when.equals:
          fields.log_type: 'database'

    #ssl.certificateAuthorities: [ "/etc/metricbeat/certs/elasticsearch-http-ca.pem" ]
#================================ Processors =====================================
# Configure processors to enhance or manipulate events generated by the beat.
processors:
  - add_host_metadata: ~
logging.level: info
logging.to_files: true
logging.files:
  path: /var/log/files
  name: filebeat
  keepfiles: 3
  permissions: 0640
2

Hello @Priya_Das

Welcome to the community!!

Could you please share the version used?

  1. We can check below if the file exists :
    ls -l /data/mysqldata/mysqld.log

  2. This file has recent data or not as ignore is 72 hours :
    /data/mysqldata/mysqld.log

  3. The user with which filebeat service is running has access to below path :
    /data/mysqldata/mysqld.log

  4. Check filebeat logs for more information related to path "/data/mysqldata/mysqld.log"
    /var/log/files/filebeat.log

Note: In the future, when adding logs or YAML content, please use the </> formatting option to enhance readability.

Thanks!!

3

filebeat-8.15.2