File Integrity Monitor Missing Events

File Integrity Monitor missed several events in a recent planned software deployment. FIM is configured to track the application folder on a dozen nearly identical hosts and was first initialized about a month ago. Other than defining some regex for our application path and excluding others, the config is mostly default and deployed using Fleet. The config is below. The stack is 8.11, FIM 1.14. 0

On some hosts, the JAR file that we are most interested in tracking shows expected activity, a delete and several attribute events that occurred in the 1.25 seconds during the copy of an updated file. Other hosts show only a delete, nothing about the new file being copied. On those hosts I did confirm, the file was picked up in the initial_scan and is in fact there, FIM missed the changes. To add to the mystery, FIM detected a wildly different number of events on all hosts. The count of events below are just our application folder impacted during the deployment:

host1 = 98
host2 = 110
host3 = 111
host4 = 104

Am I asking too much of FIM, is this a config issue, any advice to troubleshoot is appreciated!

PUT kbn:/api/fleet/package_policies/30256379-bfbd-41fe-a8b1-68ab5c0b3604
{
  "package": {
    "name": "fim",
    "version": "1.14.0"
  },
  "name": "fim-disributors",
  "namespace": "default",
  "description": "",
  "policy_id": "c13aa9c0-a9eb-11ee-a008-1718a8f5f0fc",
  "vars": {
    "paths": [
      "/bin",
      "/usr/bin",
      "/sbin",
      "/usr/sbin",
      "/etc",
      "/home/webapp/pulltabs-devel-live"
    ],
    "recursive": true,
    "scan_at_start": true,
    "hash_types": [
      "sha1",
      "md5"
    ],
    "max_file_size": "500 MiB",
    "scan_rate_per_sec": "50 MiB",
    "include_files": [],
    "exclude_files": [
      "(?i)\\.sw[nop]$",
      "~$",
      "/\\.git($|/)",
      "\\.tmp$",
      "\\.log$",
      "\\.db$",
      "/log/?$",
      "log\\.stdout",
      "dpkg-new",
      "dpkg-tmp",
      "/etc/lvm/backup",
      "/etc/lvm/archive"
    ],
    "keep_null": false,
    "tags": [
      "fim-event"
    ]
  },
  "inputs": {
    "fim-audit/file_integrity": {
      "enabled": true,
      "streams": {
        "fim.event": {
          "enabled": true
        }
      }
    }
  }
}

Found the following in the logs...

05:50:49.570
elastic_agent.auditbeat
[elastic_agent.auditbeat][warn] fsnotify watcher error
ptjava1
05:50:49.769
fim.event
failed to find message
ptjava1
05:50:50.000
system.auth
failed to find message

Can you enable debug logging on an agent and attempt to reproduce the issue?