Azure AD (modlue:o365) logs are not fetched consistently by filebeat

Elastic Stack Beats
1

Hi Team
I am New to the filebeat usage. I am trying to fetch logs from azure tenant using o365 module. I am able to get the logs sometimes but sometimes the expected logs are missing. I expect the logs when there is some tenant activity (sharepoint , admininstrative , exchange etc) . At times, the logs do show up . I tried changing the poll_interval to 30 sec (from default 3 min) with no luck
I do see activity on status:

● filebeat.service - Filebeat sends log files to Logstash or directly to Elasticsearch.
   Loaded: loaded (/usr/lib/systemd/system/filebeat.service; disabled; vendor preset: disabled)
   Active: active (running) since Wed 2022-09-14 10:53:49 IST; 16min ago
     Docs: https://www.elastic.co/beats/filebeat
 Main PID: 32031 (filebeat)
   CGroup: /system.slice/filebeat.service
           └─32031 /usr/share/filebeat/bin/filebeat --environment systemd -c /etc/filebeat/filebeat.yml --path.home /usr/share/filebeat --path.config /etc/filebeat --path.da...

Sep 14 11:09:30 crystaleye.lan filebeat[32031]: {"log.level":"debug","@timestamp":"2022-09-14T11:09:30.394+0530","log.logger":"publisher","log.origin":{"file.name"...":"1.6.0"}
Sep 14 11:09:30 crystaleye.lan filebeat[32031]: {"log.level":"debug","@timestamp":"2022-09-14T11:09:30.394+0530","log.logger":"acker","log.origin":{"file.name":"be...":"1.6.0"}
Sep 14 11:09:30 crystaleye.lan filebeat[32031]: {"log.level":"debug","@timestamp":"2022-09-14T11:09:30.394+0530","log.logger":"publisher","log.origin":{"file.name"...":"1.6.0"}
Sep 14 11:09:30 crystaleye.lan filebeat[32031]: {"log.level":"debug","@timestamp":"2022-09-14T11:09:30.394+0530","log.logger":"publisher","log.origin":{"file.name"...":"1.6.0"}
Sep 14 11:09:36 crystaleye.lan filebeat[32031]: {"log.level":"debug","@timestamp":"2022-09-14T11:09:36.268+0530","log.logger":"cfgfile","log.origin":{"file.name":"...":"1.6.0"}
Sep 14 11:09:46 crystaleye.lan filebeat[32031]: {"log.level":"debug","@timestamp":"2022-09-14T11:09:46.269+0530","log.logger":"cfgfile","log.origin":{"file.name":"...":"1.6.0"}
Sep 14 11:09:56 crystaleye.lan filebeat[32031]: {"log.level":"info","@timestamp":"2022-09-14T11:09:56.153+0530","log.logger":"monitoring","log.origin":{"file.name"...e":{"ms":1
Sep 14 11:09:56 crystaleye.lan filebeat[32031]: {"log.level":"debug","@timestamp":"2022-09-14T11:09:56.271+0530","log.logger":"cfgfile","log.origin":{"file.name":"...":"1.6.0"}
Sep 14 11:10:06 crystaleye.lan filebeat[32031]: {"log.level":"debug","@timestamp":"2022-09-14T11:10:06.271+0530","log.logger":"cfgfile","log.origin":{"file.name":"...":"1.6.0"}
Sep 14 11:10:16 crystaleye.lan filebeat[32031]: {"log.level":"debug","@timestamp":"2022-09-14T11:10:16.272+0530","log.logger":"cfgfile","log.origin":{"file.name":"...":"1.6.0"}
Hint: Some lines were ellipsized, use -l to show in full.
2

You might want to look at /var/log/filebeat/filebeat.log to get access to more logs.

3

I dont see any files in the /var/log/filebeat/
Also, say I want to collect historical logs (ex: last 1 month). Is there a way I can use filebeat to get older logs?

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.