EQL: Why basic query is different from dataset

Elastic Security SIEM
1

I'm running a simple EQL query to test as follows, but NOT reporting anything back

GET winlogbeat*/_eql/search
{
  "query": """
  process where process.name == "services.exe"
  """
}

Quite lot of data is present and If I run an sql query to see the data it is present.

POST /_sql?format=txt
{
  "query": """
  SELECT "process.name", "event.category", count(*) FROM "winlogbeat*"
  WHERE "process.name" = 'services.exe'
  GROUP BY  "process.name","event.category"
  """
} 

 process.name  |event.category |   count(*)    
---------------+---------------+---------------
services.exe   |authentication |3607           
services.exe   |process        |39

Any idea if I need to something else to setup EQL?

2

Hi @kelk! Thanks for checking out EQL! Let's see if we can get you squared away here...

The two concepts required for EQL to work are category and time, which by default correspond to the event.category and @timestamp fields, respectively.

If your mappings are correct but your documents are missing either of these fields, the search will succeed but with no results. Since you didn't mention an error, I suspect that you're missing a timestamp field.

If your documents do have a timestamp field but it's not named @timestamp, you can specify that as part of the query rather than adding/updating your documents.

I hope that helps! But if it doesn't, sharing your winlogbeat mappings along with a few representative documents and the EQL responses/errors will help to further diagnose. Cheers!

3

There was no error, but just gives empty result

{
  "is_partial" : false,
  "is_running" : false,
  "took" : 0,
  "timed_out" : false,
  "hits" : {
    "total" : {
      "value" : 0,
      "relation" : "eq"
    },
    "events" : [ ]
  }
}

btw the data comes from WinLogBeats and is parsed in ECS format too. event.category & @timestamp is present etc.

4

Strange. It worked when I put the actual index, but if I give a index* (wildcard), it fails which is a shame.

GET winlogbeat-7.9.2-2020.10.12-000001/_eql/search
{
  "query": """
  process where process.name == "services.exe"
  """
}

works !! expected it to be work with wildcard. Can you please put this as a feature?

5

@kelk glad you were able to get things working to a degree!

It's possible that there are mapping differences across your winlogbeat indexes; you might be able to narrow that down by starting with your concrete index, and then loosening the wildcard (winlogbeat-7.9.2-2020.10.12*, winlogbeat-7.9.2-2020.10*, etc) until you see the failure.

We also made an improvement or two to EQL's index resolution for 7.10, so that behavior will be improved soon!

6

thank you again.
I feel EQL have so much legs and is much simpler for aggregation/pipe/siem/event-stitch together. may be if you can develop it to the likes of Splunk SPL or similar, it would become the most important language within ELK framework

2 Likes
7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.