Filebeats not picking .log files from directories

[Dhilipkumar]

Hello All,

Im using 8.1.3 version of Elasticsearch and 8.1.0 version of filebeat for log Analysis puropse in Linux environment as a group user.
We have log files of various types (e.g .txt, .log, .html, .sql, .csv) in a directory with various sub directories say to maxdepth of 8. While running filebeat with ** in the path it picks the .html, .sql, .txt files but not .log files. The Parent directory may look like the one in the picture that i've shared.
Can someone help resolve this issue.

[warkolm (Mark Walkom)]

Welcome to our community!

It'd be useful if you could share your config.

[Dhilipkumar]

filebeat.inputs:

- type: log
  enabled: true
  paths:
    - /home/oracle/catt-logs-week23/**

# ============================== Filebeat modules ==============================

filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: true

# ======================= Elasticsearch template setting =======================

setup.template.settings:
  index.number_of_shards: 1

setup.ilm.enabled: false

# =================================== Kibana ===================================

setup.kibana:
  host: "<hostip>:5601"


# ---------------------------- Elasticsearch Output ----------------------------
output.elasticsearch:
 
  hosts: ["<hostip>:9200"]
  index: "week23-sec-last-dir-log"

setup.template:
  name: 'week23-sec-last-dir-log'
  pattern: 'week23-sec-last-dir-log'

  username: "username"
  password: "password"

# ================================= Processors =================================
processors:
  - add_host_metadata:
      when.not.contains.tags: forwarded