Webhook with variables from Query DSL hits

Elastic Security
1

Hi,
I am trying to create a rule that when triggered gets a variable, an IP in this case, from Query DSL and sends this variable through a webhook.
I should be available to get an array of these variables if there are multiple hits from the query.

The query hit would look something like this:

...
  "hits": {
    "total": {
      "value": 1,
      "relation": "XXX"
    },
    "max_score": 3,
    "hits": [
      {
          "message": """XXX""",
          "container": {
            "id": "XXX",
            "runtime": "docker",
            "image": {
              "name": "XXX"
            }
          },
          "environment": "production",
          "agent": {
            "ephemeral_id": "XXX",
            "id": "XXX",
            "name": "XXX",
            "version": "8.1.0",
            "type": "filebeat"
          },
          "status": "SUCCESS",
          "extra": {
            "user": {
              "ip": "XXX.XXX.XXX.XXX"
            },
...

So far I didn't find a way to catch the IP, I was testing this with a slack message.
I got the entire code of the hit with {{context.hits}} but it would be helpful to get just the needed values.

2

Hi @Alex_Molina_Bastos and welcome to the forum!

I'd like to learn more about what you're trying to achieve. Could you export this specific rule and share the ndjson output here?

3

Hi @georgii
The purpose is to block IP when I detect more than X attempts in the last few minutes, like more than 1000 petitions on a reset password.
The alarm calls a webhook to a bitbucket pipeline which will do whatever actions. I was wondering if it was possible to use the webhook to send the pipeline a map of values extracted from the query of the alarm.

I can't post the JSON or other sensible data, sorry

4

@Alex_Molina_Bastos If I understood your use case correctly, this is not something that you could easily do with Security rules as of now.

Firstly, we don't yet support the deduplication of alerts generated by a rule by a certain field. For instance, if a rule generates 50 alert documents during a single rule execution, and all the 50 alerts will have the same value of the host.name field, you will still get 50 alerts (all of them will be created). However, one of our teams is currently working on [Security Solution][Alerts Area] Implement alert throttling by field group · Issue #130699 · elastic/kibana · GitHub which will allow to "throttle" alerts by one or several field values. In the example above, if host.name would be specified as a throttle field, the rule would generate only one alert instead of 50.

Secondly, the mustache syntax which is used in the rule actions doesn't provide a lot of flexibility for working with json. You can't select and transform it however you want like you'd do with jq. What you can do is you can send the whole alerts json or "print" selected fields from the alerts in the text format:

Screenshot 2022-11-10 at 17.12.51
Screenshot 2022-11-10 at 17.12.512040×1296 172 KB

It would send something like this in the request body to the webhook (I truncated the JSON):

Alerts JSON:
{\"signal\":{\"depth\":1,\"original_event\":{\"category\":[\"process\"],\"id\":\"da7aadc4-3188-45cc-8cfe-776b260c5164\",\"kind\":\"event\",\"sequence\":0,\"type\":[\"start\"]},\"original_time\":\"2022-11-10T14:57:04.215Z\",\"reason\":\"process event with process iexlorer.exe, by rofwn0gwiu on Host-crfo8ihfl5 created low alert Test.\",\"rule\":{\"author\":[],\"created_at\":\"2022-11-10T13:50:41.233Z\",\"created_by\":\"custom_superuser\",\"description\":\"-\",\"enabled\":true,\"false_positives\":[],\"from\":\"now-90s\",\"id\":\"a9cd7f10-60fe-11ed-a4e9-b9430961b65e\",\"immutable\":false,\"interval\":\"30s\",\"license\":\"\",\"max_signals\":100,\"name\":\"Test\",\"references\":[],\"risk_score\":21,\"rule_id\":\"16f475c9-8074-4cc8-8107-1d819372101c\",\"severity\":\"low\",\"tags\":[],\"to\":\"now\",\"type\":\"query\",\"updated_at\":\"2022-11-10T14:56:52.296Z\",\"updated_by\":\"custom_superuser\",\"version\":2},\"status\":\"open\"},\"_id\":\"778afc216077c8b32a038162d2340c43cca365a45f42450d160f6baf118a2c1a\",\"_index\":\".internal.alerts-security.alerts-default-000001\",\"kibana\":{\"version\":\"8.6.0\",\"alert\":{\"rule\":{\"category\":\"Custom Query Rule\",\"consumer\":\"siem\",\"execution\":{\"uuid\":\"de691733-98d6-441e-a9f1-a7f5eb25be3f\"},\"name\":\"Test\",\"producer\":\"siem\",\"rule_type_id\":\"siem.queryRule\",\"uuid\":\"a9cd7f10-60fe-11ed-a4e9-b9430961b65e\",\"tags\":[],\"parameters\":{\"description\":\"-\",\"risk_score\":21,\"severity\":\"low\",\"license\":\"\",\"meta\":{\"from\":\"1m\",\"kibana_siem_app_url\":\"http://localhost:5601/kbn/app/security\"},\"author\":[],\"false_positives\":[],\"from\":\"now-90s\",\"rule_id\":\"16f475c9-8074-4cc8-8107-1d819372101c\",\"max_signals\":100,\"risk_score_mapping\":[],\"severity_mapping\":[],\"threat\":[],\"to\":\"now\",\"references\":[],\"version\":2,\"exceptions_list\":[],\"immutable\":false,\"related_integrations\":[],\"required_fields\":[],\"setup\":\"\",\"type\":\"query\",\"language\":\"kuery\",\"index\":[\"apm-*-transaction*\",\"auditbeat-*\",\"endgame-*\",\"filebeat-*\",\"logs-*\",\"packetbeat-*\",\"traces-apm*\",\"winlogbeat-*\",\"-*elastic-cloud-logs-*\"],\"query\":\"host.name: *\",\"filters\":[]},\"actions\":[{\"group\":\"default\",\"id\":\"5f388b80-6107-11ed-a4e9-b9430961b65e\",\"params\":{\"body\":\"Alerts JSON:\\n{{context.alerts}}\\n\\nHits JSON:\\n{{context.hits}}\\n\\n{{#context.alerts}}\\n- Host name: \\\"{{host.name}}\\\"\\n{{/context.alerts}}\\n\"},\"action_type_id\":\".webhook\"}],\"author\":[],\"created_at\":\"2022-11-10T13:50:41.233Z\",\"created_by\":\"custom_superuser\",\"description\":\"-\",\"enabled\":true,\"exceptions_list\":[],\"false_positives\":[],\"from\":\"now-90s\",\"immutable\":false,\"interval\":\"30s\",\"indices\":[\"apm-*-transaction*\",\"auditbeat-*\",\"endgame-*\",\"filebeat-*\",\"logs-*\",\"packetbeat-*\",\"traces-apm*\",\"winlogbeat-*\",\"-*elastic-cloud-logs-*\"],\"license\":\"\",\"max_signals\":100,\"references\":[],\"risk_score_mapping\":[],\"rule_id\":\"16f475c9-8074-4cc8-8107-1d819372101c\",\"severity_mapping\":[],\"threat\":[],\"to\":\"now\",\"type\":\"query\",\"updated_at\":\"2022-11-10T14:56:52.296Z\",\"updated_by\":\"custom_superuser\",\"version\":2,\"meta\":{\"from\":\"1m\",\"kibana_siem_app_url\":\"http://localhost:5601/kbn/app/security\"},\"risk_score\":21,\"severity\":\"low\"},\"original_time\":\"2022-11-10T14:57:04.215Z\",\"ancestors\":[{\"id\":\"lHELYoQB3uGrpQutX8EH\",\"type\":\"event\",\"index\":\".ds-logs-endpoint.events.process-default-2022.11.10-000001\",\"depth\":0}],\"status\":\"active\",\"workflow_status\":\"open\",\"depth\":1,\"reason\":\"process event with process iexlorer.exe, by rofwn0gwiu on Host-crfo8ihfl5 created low alert Test.\",\"severity\":\"low\",\"risk_score\":21,\"original_event\":{\"agent_id_status\":\"auth_metadata_missing\",\"sequence\":0,\"ingested\":\"2022-11-10T14:57:03Z\",\"kind\":\"event\",\"id\":\"da7aadc4-3188-45cc-8cfe-776b260c5164\",\"category\":[\"process\"],\"type\":[\"start\"]},\"uuid\":\"778afc216077c8b32a038162d2340c43cca365a45f42450d160f6baf118a2c1a\"},\"space_ids\":[\"default\"]},\"@timestamp\":\"2022-11-10T14:57:17.460Z\",\"agent\":{\"id\":\"57b53114-52cc-4c55-836d-ad169033de3f\",\"type\":\"endpoint\",\"version\":\"8.6.0\"},\"process\":{\"args\":[\"\\\"C:\\\\iexlorer.exe\\\"\",\"--24j\"],\"Ext\":{\"ancestry\":[]},\"group_leader\":{\"name\":\"fake leader\",\"pid\":607,\"entity_id\":\"dd2oz1bzr9\"},\"session_leader\":{\"name\":\"fake session\",\"pid\":575,\"entity_id\":\"dd2oz1bzr9\"},\"code_signature\":{\"subject_name\":\"Microsoft\",\"status\":\"trusted\"},\"entry_leader\":{\"name\":\"fake entry\",\"pid\":727,\"entity_id\":\"dd2oz1bzr9\"},\"name\":\"iexlorer.exe\",\"pid\":4958,\"working_directory\":\"/home/rofwn0gwiu/\",\"entity_id\":\"dd2oz1bzr9\",\"executable\":\"C:\\\\iexlorer.exe\",\"hash\":{\"md5\":\"bb24b5e3-0dd3-445a-9880-46cc9832edfd\"}},\"ecs\":{\"version\":\"1.4.0\"},\"data_stream\":{\"namespace\":\"default\",\"type\":\"logs\",\"dataset\":\"endpoint.events.process\"},\"host\":{\"hostname\":\"Host-crfo8ihfl5\",\"os\":{\"Ext\":{\"variant\":\"Windows Server Release 2\"},\"name\":\"Windows\",\"family\":\"windows\",\"version\":\"6.3\",\"platform\":\"Windows\",\"full\":\"Windows Server 2012R2\"},\"ip\":[\"10.109.146.91\"],\"name\":\"Host-crfo8ihfl5\",\"id\":\"675b73c2-9b5d-41f9-a9f5-3a72b747dd02\",\"mac\":[\"9f-e9-76-28-31-5f\"],\"architecture\":\"4d5uenqhyo\"},\"user\":{\"domain\":\"v38igs0p60\",\"name\":\"rofwn0gwiu\"},\"event\":{\"agent_id_status\":\"auth_metadata_missing\",\"sequence\":0,\"ingested\":\"2022-11-10T14:57:03Z\",\"kind\":\"signal\",\"id\":\"da7aadc4-3188-45cc-8cfe-776b260c5164\",\"category\":[\"process\"],\"type\":[\"start\"]}},{\"signal\":{\"depth\":1,\"original_event\":{\"category\":[\"process\"],\"id\":\"20e21870-5024-466b-b50e-ceca144a822f\",\"kind\":\"event\",\"sequence\":1,\"type\":[\"start\"]},\"original_time\":\"2022-11-10T14:57:05.215Z\",\"reason\":\"process event with process powershell.exe, by 2qv98zp1y0 on Host-crfo8ihfl5 created low alert Test.\",\"rule\":{\"author\":[],\"created_at\":\"2022-11-10T13:50:41.233Z\",\"created_by\":\"custom_superuser\",\"description\":\"-\",\"enabled\":true,\"false_positives\":[],\"from\":\"now-90s\",\"id\":\"a9cd7f10-60fe-11ed-a4e9-b9430961b65e\",\"immutable\":false,\"interval\":\"30s\",\"license\":\"\",\"max_signals\":100,\"name\":\"Test\",\"references\":[],\"risk_score\":21,\"rule_id\":\"16f475c9-8074-4cc8-8107-1d819372101c\",\"severity\":\"low\",\"tags\":[],\"to\":\"now\",\"type\":\"query\",\"updated_at\":\"2022-11-10T14:56:52.296Z\",\"updated_by\":\"custom_superuser\",\"version\":2},\"status\":\"open\"},\"_id\":\"9b81e0fb63958748ee216ad10884aba25496db2999d213e8961a296bc0e815f4\",\"_index\":\".internal.alerts-security.alerts-default-000001\",\"kibana\":{\"version\":\"8.6.0\",\"alert\":{\"rule\":{\"category\":\"Custom Query Rule\",\"consumer\":\"siem\",\"execution\":{\"uuid\":\"de691733-98d6-441e-a9f1-a7f5eb25be3f\"},\"name\":\"Test\",\"producer\":\"siem\",\"rule_type_id\":\"siem.queryRule\",\"uuid\":\"a9cd7f10-60fe-11ed-a4e9-b9430961b65e\",\"tags\":[],\"parameters\":{\"description\":\"-\",\"risk_score\":21,\"severity\":\"low\",\"license\":\"\",\"meta\":{\"from\":\"1m\",\"kibana_siem_app_url\":\"http://localhost:5601/kbn/app/security\"},\"author\":[],\"false_positives\":[],\"from\":\"now-90s\",\"rule_id\":\"16f475c9-8074-4cc8-8107-1d819372101c\",\"max_signals\":100,\"risk_score_mapping\":[],\"severity_mapping\":[],\"threat\":[],\"to\":\"now\",\"references\":[],\"version\":2,\"exceptions_list\":[],\"immutable\":false,\"related_integrations\":[],\"required_fields\":[],\"setup\":\"\",\"type\":\"query\",\"language\":\"kuery\",\"index\":[\"apm-*-transaction*\",\"auditbeat-*\",\"endgame-*\",\"filebeat-*\",\"logs-*\",\"packetbeat-*\",\"traces-apm*\",\"winlogbeat-*\",\"-*elastic-cloud-logs-*\"],\"query\":\"host.name: *\",\"filters\":[]},\"actions\":[{\"group\":\"default\",\"id\":\"5f388b80-6107-11ed-a4e9-b9430961b65e\",\"params\":{\"body\":\"Alerts JSON:\\n{{context.alerts}}\\n\\nHits JSON:\\n{{context.hits}}\\n\\n{{#context.alerts}}\\n- Host name: \\\"{{host.name}}\\\"\\n{{/context.alerts}}\\n\"},\"action_type_id\":\".webhook\"}],\"author\":[],\"created_at\":\"2022-11-10T13:50:41.233Z\",\"created_by\":\"custom_superuser\",\"description\":\"-\",\"enabled\":true,\"exceptions_list\":[],\"false_positives\":[],\"from\":\"now-90s\",\"immutable\":false,\"interval\":\"30s\",\"indices\":[\"apm-*-transaction*\",\"auditbeat-*\",\"endgame-*\",\"filebeat-*\",\"logs-*\",\"packetbeat-*\",\"traces-apm*\",\"winlogbeat-*\",\"-*elastic-cloud-logs-*\"],\"license\":\"\",\"max_signals\":100,\"references\":[],\"risk_score_mapping\":[],\"rule_id\":\"16f475c9-8074-4cc8-8107-1d819372101c\",\"severity_mapping\":[],\"threat\":[],\"to\":\"now\",\"type\":\"query\",\"updated_at\":\"2022-11-10T14:56:52.296Z\",\"updated_by\":\"custom_superuser\",\"version\":2,\"meta\":{\"from\":\"1m\",\"kibana_siem_app_url\":\"http://localhost:5601/kbn/app/security\"},\"risk_score\":21,\"severity\":\"low\"},\"original_time\":\"2022-11-10T14:57:05.215Z\",\"ancestors\":[{\"id\":\"lXELYoQB3uGrpQutX8EH\",\"type\":\"event\",\"index\":\".ds-logs-endpoint.events.process-default-2022.11.10-000001\",\"depth\":0}],\"status\":\"active\",\"workflow_status\":\"open\",\"depth\":1,\"reason\":\"process event with process powershell.exe, by 2qv98zp1y0 on Host-crfo8ihfl5 created low alert Test.\",\"severity\":\"low\",\"risk_score\":21,\"original_event\":{\"agent_id_status\":\"auth_metadata_missing\",\"sequence\":1,\"ingested\":\"2022-11-10T14:57:03Z\",\"kind\":\"event\",\"id\":\"20e21870-5024-466b-b50e-ceca144a822f\",\"category\":[\"process\"],\"type\":[\"start\"]},\"uuid\":\"9b81e0fb63958748ee216ad10884aba25496db2999d213e8961a296bc0e815f4\"},\"space_ids\":[\"default\"]},\"@timestamp\":\"2022-11-10T14:57:17.461Z\",\"agent\":{\"id\":\"57b53114-52cc-4c55-836d-ad169033de3f\",\"type\":\"endpoint\",\"version\":\"8.6.0\"},\"process\":{\"Ext\":{\"ancestry\":[\"dd2oz1bzr9\"]},\"parent\":{\"pid\":4958,\"entity_id\":\"dd2oz1bzr9\"},\"group_leader\":{\"name\":\"fake leader\",\"pid\":453,\"entity_id\":\"dd2oz1bzr9\"},\"pid\":3660,\"working_directory\":\"/home/2qv98zp1y0/\",\"entity_id\":\"6nknbziq0x\",\"executable\":\"C:\\\\powershell.exe\",\"args\":[\"\\\"C:\\\\powershell.exe\\\"\",\"--8h4\"],\"session_leader\":{\"name\":\"fake session\",\"pid\":520,\"entity_id\":\"dd2oz1bzr9\"},\"code_signature\":{\"subject_name\":\"Microsoft\",\"status\":\"trusted\"},\"entry_leader\":{\"name\":\"fake entry\",\"pid\":337,\"entity_id\":\"dd2oz1bzr9\"},\"name\":\"powershell.exe\",\"hash\":{\"md5\":\"882a7b31-f717-424c-a437-dd19c3b45494\"}},\"ecs\":{\"version\":\"1.4.0\"},\"data_stream\":{\"namespace\":\"default\",\"type\":\"logs\",\"dataset\":\"endpoint.events.process\"},\"host\":{\"hostname\":\"Host-crfo8ihfl5\",\"os\":{\"Ext\":{\"variant\":\"Windows Server Release 2\"},\"name\":\"Windows\",\"family\":\"windows\",\"version\":\"6.3\",\"platform\":\"Windows\",\"full\":\"Windows Server 2012R2\"},\"ip\":[\"10.109.146.91\"],\"name\":\"Host-crfo8ihfl5\",\"id\":\"675b73c2-"

- Host name: "Host-crfo8ihfl5"
- Host name: "Host-crfo8ihfl5"
- Host name: "Host-crfo8ihfl5"
- Host name: "Host-crfo8ihfl5"
- Host name: "Host-crfo8ihfl5"
- Host name: "Host-crfo8ihfl5"
- Host name: "Host-crfo8ihfl5"
- Host name: "Host-crfo8ihfl5"
- Host name: "Host-rcsgxu4c5c"
- Host name: "Host-crfo8ihfl5"
- Host name: "Host-rcsgxu4c5c"
- Host name: "Host-crfo8ihfl5"
- Host name: "Host-rcsgxu4c5c"
- Host name: "Host-crfo8ihfl5"

Then, on the receiving side of the webhook connector (e.g. a bitbucket pipeline) you could probably implement parsing of these request bodies and deduplication of the field values, if the receiving side allows doing that.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.