Include Fields in Watcher Email Alert

alerting

(G T) #1

Hi there,

I have set up an advanced Watch and it triggers upon 3 or more of the same TargetUserName with the EventID:4625, basically that alerts on any user failing to logon 3 times in 3 minutes. Here is my body section of the email callback:

"body": {
          "text": "The user: {{ctx.payload.TargetUserName}} failed to logon: {{ctx.payload.hits.total}} times."
        }

The second ctx call works and I am shown the amount of times the TargetUserName field appeard, but as for the: {{ctx.payload.TargetUserName}} no data shows, I presume I am doing something wrong here?

Cheers,

G

Below is the entire Watch, just in case anyone needs it.

{
  "trigger": {
    "schedule": {
      "interval": "1m"
    }
  },
  "input": {
    "search": {
      "request": {
        "search_type": "query_then_fetch",
        "indices": [
          "*"
        ],
        "types": [],
        "body": {
          "size": 0,
          "query": {
            "bool": {
              "filter": [
                {
                  "range": {
                    "@timestamp": {
                      "gte": "now-5m",
                      "lte": "now"
                    }
                  }
                },
                {
                  "term": {
                    "EventID": "4625"
                  }
                }
              ]
            }
          },
          "aggs": {
            "User": {
              "terms": {
                "field": "TargetUserName.keyword"
              }
            }
          }
        }
      }
    }
  },
  "condition": {
    "compare": {
      "ctx.payload.aggregations.User.buckets.0.doc_count": {
        "gte": 3
      }
    }
  },
  "actions": {
    "email_admin": {
      "throttle_period_in_millis": 50000,
      "email": {
        "profile": "standard",
        "to": [
          "xxxxxxxxxxxx@mycompany.com"
        ],
        "subject": "Warning: Multiple Failed Logons",
        "body": {
          "text": "The user: {{ctx.payload.TargetUserName}} failed to logon: {{ctx.payload.hits.total}} times."
        }
      }
    }
  }
}

(Alexander Reelsen) #2

can you please include the output of the execute watch API as well? Thanks.


(G T) #3

Hi @spinscale could you instruct me on where to find the output of the execute watch API?

Cheers,

G


(Alexander Reelsen) #4

Hey,

it is a dedicated API that captures the whole watch execution output and simplifies debugging tremendously.

See the docs at https://www.elastic.co/guide/en/elasticsearch/reference/6.1/watcher-api-execute-watch.html

--Alex


(G T) #5

Here you are:

{
  "_id": "W1234_b2b77b91-c0f7-4ee2-9970-3aa41112bc59-2018-01-16T14:06:36.040Z",
  "watch_record": {
    "watch_id": "W1234",
    "node": "uy-aQe2PT12NAjCj-B7QKQ",
    "state": "executed",
    "status": {
      "state": {
        "active": true,
        "timestamp": "2018-01-16T14:04:36.002Z"
      },
      "last_checked": "2018-01-16T14:06:36.040Z",
      "last_met_condition": "2018-01-16T14:06:36.040Z",
      "actions": {
        "email_admin": {
          "ack": {
            "timestamp": "2018-01-16T14:05:36.403Z",
            "state": "ackable"
          },
          "last_execution": {
            "timestamp": "2018-01-16T14:06:36.040Z",
            "successful": true
          },
          "last_successful_execution": {
            "timestamp": "2018-01-16T14:06:36.040Z",
            "successful": true
          }
        }
      },
      "execution_state": "executed",
      "version": 23129
    },
    "trigger_event": {
      "type": "manual",
      "triggered_time": "2018-01-16T14:06:36.040Z",
      "manual": {
        "schedule": {
          "scheduled_time": "2018-01-16T14:06:36.040Z"
        }
      }
    },
    "input": {
      "search": {
        "request": {
          "search_type": "query_then_fetch",
          "indices": [
            "*"
          ],
          "types": [],
          "body": {
            "size": 0,
            "query": {
              "bool": {
                "filter": [
                  {
                    "range": {
                      "@timestamp": {
                        "gte": "now-5m",
                        "lte": "now"
                      }
                    }
                  },
                  {
                    "term": {
                      "EventID": "4625"
                    }
                  }
                ]
              }
            },
            "aggs": {
              "User": {
                "terms": {
                  "field": "TargetUserName.keyword"
                }
              }
            }
          }
        }
      }
    },
    "condition": {
      "compare": {
        "ctx.payload.aggregations.User.buckets.0.doc_count": {
          "gte": 3
        }
      }
    },
    "metadata": {
      "name": "Failed User Logons",
      "xpack": {
        "type": "json"
      }
    },
    "result": {
      "execution_time": "2018-01-16T14:06:36.040Z",
      "execution_duration": 2347,
      "input": {
        "type": "search",
        "status": "success",
        "payload": {
          "_shards": {
            "total": 122,
            "failed": 0,
            "successful": 122,
            "skipped": 0
          },
          "hits": {
            "hits": [],
            "total": 38,
            "max_score": 0
          },
          "took": 12,
          "timed_out": false,
          "aggregations": {
            "User": {
              "doc_count_error_upper_bound": 0,
              "sum_other_doc_count": 0,
              "buckets": [
                {
                  "doc_count": 38,
                  "key": "MyNameGoesHere"
                }
              ]
            }
          }
        },
        "search": {
          "request": {
            "search_type": "query_then_fetch",
            "indices": [
              "*"
            ],
            "types": [],
            "body": {
              "size": 0,
              "query": {
                "bool": {
                  "filter": [
                    {
                      "range": {
                        "@timestamp": {
                          "gte": "now-5m",
                          "lte": "now"
                        }
                      }
                    },
                    {
                      "term": {
                        "EventID": "4625"
                      }
                    }
                  ]
                }
              },
              "aggs": {
                "User": {
                  "terms": {
                    "field": "TargetUserName.keyword"
                  }
                }
              }
            }
          }
        }
      },
      "condition": {
        "type": "compare",
        "status": "success",
        "met": true,
        "compare": {
          "resolved_values": {
            "ctx.payload.aggregations.User.buckets.0.doc_count": 38
          }
        }
      },
      "actions": [
        {
          "id": "email_admin",
          "type": "email",
          "status": "success",
          "email": {
            "account": "outlook_account",
            "message": {
              "id": "W1234_b2b77b91-c0f7-4ee2-9970-3aa41112bc59-2018-01-16T14:06:36.040Z",
              "sent_date": "2018-01-16T14:06:36.054Z",
              "to": [
                "yournamegoeshere@myawesomecompany.com"
              ],
              "subject": "Warning: Multiple Failed Logons",
              "body": {
                "text": "The user:  failed to logon: 38 times."
              }
            }
          }
        }
      ]
    },
    "messages": []
  }
}

Also please not, I have changed the: The user: {{ctx.payload.TargetUserName}} failed to logon to
The user: {{ctx.payload.TargetUserName.keyword}} failed to logon but that didn't work.

If you need any more info please do no hesitate to ask :slight_smile:

Cheers,

G


(Alexander Reelsen) #6

Check out the result.input.payload object from the execute watch API. It does not contain the any field named TargetUserName and thus is why that is empty.

You can only access fields from that payload in your mustache templates or your scripts.


(G T) #7

How would I go about including TargetUserName.keyword?

Cheers,

G


Watcher payload json array access
(Alexander Reelsen) #8

judging from your aggregation you want to include ctx.payload.aggregations.User.buckets and every key from there.

Try this in your email body as a starter

{{#ctx.payload.aggregations.User.buckets}}{{key}}{{/ctx.payload.aggregations.User.buckets}}

(G T) #9

@spinscale Hi, thank you for that, it worked! I'm not really sure how I would have got there by myself, this seems quite confusing. Could you explain how I could have found: {{#ctx.payload.aggregations.User.buckets}}{{key}}{{/ctx.payload.aggregations.User.buckets}} on my own?

Cheers,

G


(Alexander Reelsen) #10

Hey,

it's actually no rocket science. One of the reason I was asking so hard for the execute watch API output is the fact, that you can see in the input.result.payload field the whole data structure that you are able to access.

As you can see, there is no TargetUserName.keyword.

This blog post might help to establish a solid workflow when writing and developing watches:

--Alex


(G T) #11

@spinscale Just read through that and it seems really useful, so thanks!

I believe that I need to use the extract function to saves certain fields when the query if run. I will post below the watch I now have, I have the issue of no know where to put the extract function.

{
  "trigger": {
    "schedule": {
      "interval": "1m"
    }
  },
  "input": {
    "search": {
      "request": {
        "search_type": "query_then_fetch",
        "indices": [
          "*"
        ],
        "types": [],
        "body": {
          "size": 0,
          "query": {
            "bool": {
              "filter": [
                {
                  "range": {
                    "@timestamp": {
                      "gte": "now-1m",
                      "lte": "now"
                    }
                  }
                },
                {
                  "term": {
                    "EventID": [
                      "4728",
                      "4732",
                      "4756"
                    ]
                  }
                }
              ]
            }
          }
        }
      }
    }
  },
  "condition": {
    "compare": {
      "ctx.payload.aggregations.User.buckets.0.doc_count": {
        "gte": 1
      }
    }
  },
  "actions": {
    "email_admin": {
      "throttle_period_in_millis": 50000,
      "email": {
        "profile": "standard",
        "to": [
          "email@email.com"
        ],
        "subject": "Warning: User Added to group",
        "body": {
          "text": "A user has been added to the following group: {{ctx.payload.TargetUserName}}"
        }
      }
    }
  }
} 

I would like to extract the TargetUserName field, I am unsure where to add:
"extract": ["TargetUserName"]

Please could you provide some guidance? I feel that I am almost there with these watches!

Cheers,

G


(Alexander Reelsen) #12

Hey,

using extract is purely optional. All it does is reducing the payload, but you already have everything available.

if you want to use it, see https://www.elastic.co/guide/en/x-pack/6.1/input-search.html#_extracting_specific_fields

--Alex


(G T) #13

@spinscale I have read through that and am still confused. I don't understand why when I execute the watch, there are no fields returned. I would have thought it the watch runs the query:
"EventID = 4625" then it would return the data from the logs that include this EventID?

Cheers,

G


(Alexander Reelsen) #14

you explicitely configured size: 0 in your query, so you are not getting back any hits


(G T) #15

Oh okay, I had just copied a template watch and worked from there. I think this was where I was going wrong. I seem to have everything else pretty much sorted watch wise, I just needed to look at returning the values from the query.

Thank you so much for all the help you have given me, it is much appreciated.

Cheers,

G


(Gopal) #16

Maybe this needs to be separate entries.

What's the best way to include specific fields from all ctx.payload.hits.hits in a tabular format within email boxy? Making it easy to read for end-users.


(Alexander Reelsen) #17

as that is a completely new topic, making it separate makes a ton of sense.

You could use an html body in an email and make use of a table.

You can loop through the hits like this

{#ctx.payload.hits.hits}* {_source.my_field}{/ctx.payload.hits.hits}

(G T) #18

Hi @spinscale,

I have had chance to get look into advanced watches again.

I am now getting values returned after looking at your advice.

"hits": {
          "hits": [
            {
              "_index": "main",
              "_type": "doc",
              "_source": {
                "process_id": 620,
                "computer_name": "TEST",
                "keywords": [
                  "Audit Failure"
                ],
                "log_name": "Security",
                "level": "Information",
                "record_number": "57628",
                "event_data": {
                  "Status": "0xc000006d",
                  "ProcessName": "-",
                  "LogonType": "3",
                  "IpPort": "0",
                  "SubjectLogonId": "0x0",
                  "TransmittedServices": "-",
                  "KeyLength": "0",
                  "LmPackageName": "-",
                  "SubjectUserName": "-",
                  "FailureReason": "%%2313",
                  "WorkstationName": "TEST",
                  "SubjectDomainName": "-",
                  "IpAddress": "TESTIP",
                  "ProcessId": "0x0",
                  "SubStatus": "0xc000006a",
                  "TargetDomainName": "TEST",
                  "LogonProcessName": "NtLmSsp ",
                  "SubjectUserSid": "S-1-0-0",
                  "TargetUserSid": "S-1-0-0",
                  "AuthenticationPackageName": "NTLM"
                },
                "opcode": "Info",
                "tags": [
                  "beats_input_codec_plain_applied"
                ],
                "thread_id": 496,
                "@timestamp": "2018-02-05T10:33:50.722Z",
                "task": "Logon",
                "event_id": 4625,
                "provider_guid": "{54849625-5478-4994-A5BA-3E3B0328C30D}",
                "activity_id": "{2BF93E71-95E4-0001-8B3E-F92BE495D301}",
                "beat": {
                  "hostname": "TEST",
                  "name": "TEST",
                  "version": "5.6.3"
                },
                "host": "TEST",
                "target_username": "TestUser",
                "source_name": "Microsoft-Windows-Security-Auditing"

Above is the output from the watch query, I have set up the email body:

    "actions": {
"email_admin": {
  "throttle_period_in_millis": 50000,
  "email": {
    "profile": "standard",
    "to": [
      "george.townson@commissum.com"
    ],
    "subject": "Warning: User Added to Security Enabled Group",
    "body": {
      "text": "A user was added to a security enabled group. {{#ctx.payload.hits.hits}} {{_source.name.target_username}} {{/ctx.payload.hits.hits}}"
    }
  }
}

}

Currently I see: "A user was added to a security enabled group. " as the output. How would I get the email callback to return every "target_username", so I can see which users failed to login?

cheers,

G


(G T) #19

I've just worked it out. Looked back at your example and realized I had 'name.' in the section of code that returns my 'target_username' field.

Original non working:

    "subject": "Warning: User Added to Security Enabled Group",
        "body": {
          "text": "A user was added to a security enabled group. {{#ctx.payload.hits.hits}} {{_source.name.target_username}} {{/ctx.payload.hits.hits}}"
        }

New code now working:

"subject": "Warning: User Added to Security Enabled Group",
    "body": {
      "text": "A user was added to a security enabled group. {{#ctx.payload.hits.hits}} {{_source.target_username}} {{/ctx.payload.hits.hits}}"
    }

Hope this helps someone,

Cheers,

George


(system) #20

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.