Getting only items from "array_compare" condition is met

Elastic Stack Elasticsearch
1

Hello Everyone,

I am trying to get IP-address that is been used to register more that 2 users and flag the IP as anomolous.
I tried to do "aggs" by grouping by IPaddress with the "array_compare" condition.
Once the condition is met on some of my array values, I need the related action to take only those values and not all of them for reporting purposes.

Condition:

 "condition": {
    "array_compare": {
      "ctx.payload.aggregations.group_by_ip.buckets": {
        "path": "doc_count",
        "gte": {
          "value": 2,
          "quantifier": "some"
        }
      }
    }
  }

Here is my action:

"actions": {
    "email_administrator": {
        "email": {
        "profile": "standard",
        "from": "'****'",
        "priority": "high",
        "to": [
          "'***'"
        ],
        "subject": "Model - Encountered Multiple SUR - Successful User Registration",
        "body": {
          "text": "The IP [{{#ctx.payload.aggregations.group_by_ip.buckets}}{{key}} {{/ctx.payload.aggregations.group_by_ip.buckets}}] spiked usage with [{{#ctx.payload.aggregations.group_by_ip.buckets}}{{doc_count}} {{/ctx.payload.aggregations.group_by_ip.buckets}}]"
        }
      }
    }
  }

I see from other posts that I need to do script transform that does this filtering. Can anyone please provide me sample of how to do that?

Appreciate your help!

Thanks!
SV

2

Hey,

you need to do another script transform here, something like

ctx.payload.aggregations.group_by_ip.buckets.stream().filter(b -> b.doc_count > 2).collect(Collectors.toList));

however some aggregations also support the min_doc_count parameter, which could be used instead, allowing you to check if there are any buckets at all and not needing to do any manual filtering.

--Alex

3

Hello Alex,

Appreciate your response. This is one awesome forum!!!

I was able to a transform like below.

  "transform": {
        "script": {
          "source": "List l = new ArrayList(); int count=1; for(item in ctx.payload.aggregations.group_by_email.buckets) { if(item.doc_count>1){l.add(item); count++;}} return ['user_info':l];",
          "lang": "painless"
        }
      }

I know this overwrites the original payload, I need some information from original payload.

This transform will get me Ip-addresses that is been used to register more than 2 users.The transformed payload only has IPaddress and doc_count, now I need email_address for these records that is there in original payload.

How can I get the information in the original payload along with the aggregations payload?

Thanks!
SV

4

hey,

you can just attach the original payload as well, like

x = ctx.payload;
x.user_info = l;
return x;
5

Hey Alex,

That worked. Thanks so much.

SV

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.