Filebeat when started as homebrew service is not harvesting logs

[Venkat_Ram_Movva]

Hi,

I installed filebeat-full using the brew command from the documentation. I changed the config file(present at /usr/local/etc/filebeat/filebeat.yml). I started the service using brew services start filebeat-full. But I am not seeing logs getting harvested and pushed into Kafka. Checked the logs at /usr/local/var/log/filebeat. There are no errors.

But when I start as a foreground process using filebeat -e -d "*" , I am able to see that the events are getting published. And I am able to see them in Kibana.

I am interested in running this as a daemon. I don't want to have to start this manually each time a mac restarts or if someone accidentally stops this process.

Mac OS version: 10.15.5
Filebeat version from homebrew: 7.8.0

Filebeat configuration file :

        #=========================== Filebeat prospectors =============================

    filebeat.inputs:
        -
          paths:
            - /Users/myuser/Desktop/Nightly/a/*.log
          ignore_older: 48h
          close_inactive: 5m
          clean_inactive: 49h
          multiline.pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}'
          multiline.negate: true
          multiline.match: after
          processors:
          - add_fields:
              fields:
                kafka_topic: mykafkatopic
    #================================ Outputs =====================================

    #------------------------------- Kafka output ----------------------------------
    output.kafka:
      enabled: true
      hosts: ["Kafka.mydomain.COM:9092"]
      topic: '%{[fields.kafka_topic]}'
      partition.hash:
        reachable_only: false

    #================================ Logging =====================================

    logging:
      to_files: true
      files:
        rotateeverybytes: 10485760 # = 10MB
      selectors: ["*"]
      level: debug

Please help me in debugging this issue.

Thanks,
Venkat.