Verification_exception Error during Rule Execution

[inf]

Hi there,

I am facing a particular error regarding many pre-built Elastic rules. Once they were activated, many of them start failing when executing - with the error: An error occurred during rule execution: message: "verification_exception".

Some of these rules include:

• Suspicious .NET Code Compilation
• Suspicious WMI Image Load from MS Office
• NTDS or SAM Database File Copied
• Lateral Movement via Startup Folder

It appears that these rules' queries are written in EQL, and that they typically have 3 index patterns: winlogbeat-*, logs-endpoint.events.*, and logs-windows.*. I have only got Winlogbeat in my case, without Elastic Agents/Endpoints running.

Must I have all 3 of the index patterns present in order for these rules to execute successfully?

For reference, a slight variation of this issue is found in this GitHub issue.

[inf]

Bump!

[Frank_Hassanabad]

Hi there,

Usually that error means you do not have valid EQL because of a missing data driven value.

For example in dev tools if you do something like this:

POST /auditbeat-*/_eql/search
{
  "query": "sequence by blah.name [any where true] [any where true]"
}

You get this error back:

{
  "error" : {
    "root_cause" : [
      {
        "type" : "verification_exception",
        "reason" : "Found 1 problem\nline 1:13: Unknown column [blah.name], did you mean any of [vlan.name, dll.name, file.name, geo.name, host.name, os.name, rule.name, user.name, agent.name, group.name, package.name]?"
      }
    ],
    "type" : "verification_exception",
    "reason" : "Found 1 problem\nline 1:13: Unknown column [blah.name], did you mean any of [vlan.name, dll.name, file.name, geo.name, host.name, os.name, rule.name, user.name, agent.name, group.name, package.name]?"
  },
  "status" : 400
}

I would if possible take your query and set of indices and play with them in the dev tooling to see if you can get the reason back if you're not in another part of the UI. Should help you sleuth it out.

[Frank_Hassanabad]

You should be able to see the same validation by clicking in the UI as well:

If you're making a detection engine rule which should show the same type of errors.