EQL sequence query returns strange result

Elastic Stack Elasticsearch
1

Hello,

I have an EQL sequence query in ElasticSearch 7.11.1 that returns strange results: the join_keys value is different from the user.email that is in the results.
Here is the query:

GET /my_index/_eql/search?filter_path=hits.sequences.join_keys,hits.sequences.events._source.@timestamp,hits.sequences.events._source.user.email,hits.sequences.events._source.source.ip
{
  "query": """
    sequence by user.email
      [ web where url.path != "/reset" ]
      [ web where stringContains(url.path, "download") ]
  """,
  "size": 1000
}

And a sample of the results:

  "hits" : {
    "sequences" : [
      {
        "join_keys" : [
          "user1@domain1.com"
        ],
        "events" : [
          {
            "_source" : {
              "source" : {
                "ip" : "redacted"
              },
              "@timestamp" : "2021-05-02T23:25:11.781Z",
              "user" : {
                "email" : "user2@domain2.com"
              }
            }
          },
          {
            "_source" : {
              "source" : {
                "ip" : "redacted"
              },
              "@timestamp" : "2021-05-02T23:25:22.065Z",
              "user" : {
                "email" : "user2@domain2.com"
              }
            }
          }
        ]
      },

Is this behavior normal ?
Thanks
Antoine

2

Ok I see what I am doing wrong here, I should not use the != operator here.
What I'm trying to achieve here is to list the users for which I see downloads unless I see that they used the reset page first.
Is it possible with EQL ?

3

I did some more tests and there is definitely something wrong with the EQL sequence query output. I filtered on 2 users for a test and removed the negation and the results are inconsistent with the join_keys being different from the user.email.

GET /my_index/_eql/search?filter_path=hits.sequences.join_keys,hits.sequences.events._source.@timestamp,hits.sequences.events._source.user.email,hits.sequences.events._source.source.ip
{
  "query": """
    sequence by user.email
      [ web where url.path == "/reset"]
      [ web where url.path: ("*download*") ] 
  """,
  "filter": {
    "bool": {
        "should": [
          {
            "match_phrase": {
              "user.email": "user1@domain1.com"
            }
          },
          {
            "match_phrase": {
              "user.email": "user2@domain2.com"
            }
          }
        ],
        "minimum_should_match": 1
      }
  },
  "size": 1000
}

Result:

{
  "hits" : {
    "sequences" : [
      {
        "join_keys" : [
          "user1@domain1.com"
        ],
        "events" : [
          {
            "_source" : {
              "source" : {
                "ip" : "redacted"
              },
              "@timestamp" : "2021-04-16T00:02:07.970Z",
              "user" : {
                "email" : "user2@domain2.com"
              }
            }
          },
          {
            "_source" : {
              "source" : {
                "ip" : "redacted"
              },
              "@timestamp" : "2021-04-20T16:52:18.660Z",
              "user" : {
                "email" : "user2@domain2.com"
              }
            }
          }
        ]
      },
      {
        "join_keys" : [
          "user2@domain2.com"
        ],
        "events" : [
          {
            "_source" : {
              "source" : {
                "ip" : "redacted"
              },
              "@timestamp" : "2021-05-10T08:15:03.777Z",
              "user" : {
                "email" : "user1@domain1.com"
              }
            }
          },
          {
            "_source" : {
              "source" : {
                "ip" : "redacted"
              },
              "@timestamp" : "2021-05-10T08:17:03.559Z",
              "user" : {
                "email" : "user1@domain1.com"
              }
            }
          }
        ]
      }
    ]
  }
}

I think I will open an issue on GitHub as I have not seen any related to that problem.

4

Thanks for opening the issue!

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.