# Calling Alerts from Watchers to detection Signals

**URL:** <https://discuss.elastic.co/t/calling-alerts-from-watchers-to-detection-signals/250032>\
**Category:** SIEM\
**Created:** [September 25, 2020, 10:12pm UTC](https://discuss.elastic.co/t/calling-alerts-from-watchers-to-detection-signals/250032 "2020-09-25T22:12:57Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![Suro](https://avatars.discourse-cdn.com/v4/letter/s/b9bd4f/32.png) [@Suro](https://discuss.elastic.co/u/Suro)\
**Post date:** [September 25, 2020, 10:12pm UTC](https://discuss.elastic.co/t/calling-alerts-from-watchers-to-detection-signals/250032/1 "2020-09-25T22:12:58Z")

</div>

Hi,  
We are running multiple watchers and as the SIEM detection does not provide the level of customizations, I have been unable to create use cases which takes aggregation for example under consideration. So I want to create a rule in SIEM that it has to check the alerts index every 5 mins and generate a signal with the alerts observed.

Something that I am trying - \_index: alerts\*. The Signal Rule shows successfully executed but no events gets generated even when the alerts index is getting the alerts on time.

---

<div class="post-metadata">

**Author:** ![Frank\_Hassanabad](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/frank_hassanabad/32/49255_2.png) [@Frank\_Hassanabad](https://discuss.elastic.co/u/Frank_Hassanabad)\
**Post date:** [September 25, 2020, 11:35pm UTC](https://discuss.elastic.co/t/calling-alerts-from-watchers-to-detection-signals/250032/2 "2020-09-25T23:35:39Z")

</div>

We do currently have threshold rules that might help you out?

 ![Screen Shot 2020-09-25 at 5.34.59 PM](https://us1.discourse-cdn.com/elastic/original/3X/3/9/392e6abce663608245c940591fd4f6ac95235210.png)

But if not your current issue could be that the alert index is not compatible with the ECS (Elastic Common Schema) and then cannot copy over the signal.

Do you have the mapping of the alert index you are trying to create a signal from? I can take a look for you. We have improved the error messaging in our next upcoming release to report more errors transparently to the UI from this release fwiw:

> <https://github.com/elastic/kibana/pull/77687>

  

> <https://github.com/elastic/kibana/pull/78004>

---

<div class="post-metadata">

**Author:** ![borna\_talebi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/borna_talebi/32/76317_2.png) [@borna\_talebi](https://discuss.elastic.co/u/borna_talebi)\
**Post date:** [September 26, 2020, 4:46am UTC](https://discuss.elastic.co/t/calling-alerts-from-watchers-to-detection-signals/250032/3 "2020-09-26T04:46:23Z")

</div>

Hi,  
Did you add the index pattern into siem default indices? (via `stack management->kibana-> advanced setting`)  
If you did, it should be available in the index patterns section when you want to create a new rule.  
also, try a different query like `somefield : *` and see if it works.

---

<div class="post-metadata">

**Author:** ![Suro](https://avatars.discourse-cdn.com/v4/letter/s/b9bd4f/32.png) [@Suro](https://discuss.elastic.co/u/Suro)\
**Post date:** [September 26, 2020, 10:37am UTC](https://discuss.elastic.co/t/calling-alerts-from-watchers-to-detection-signals/250032/4 "2020-09-26T10:37:54Z")

</div>

I have added alerts\* under default indices and still not working -

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/7/d/7d6f4067f9ec80ca6481cac1c7d6dc9b93ff5fe1.png)

Here is the use case -

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/2/d/2de9a250d3a87988d5fc0581433581ba75bebfe1.png)

When I run the same query in Discover Kibana, I get the events, but the Detection rule is not letting me do it.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/1/2/12ae07a717ebbb368ccba9fadd16e9bd5d0a9faf.png)

I would need help with the other suggestions from Frank -  
a) How do I check the mapping of the index. This is what I saw, let me know if this helps -

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/d/b/db459502e25fbb621dd609a9cfe7d39a7d615f0e.png)

b) The option for threshold that is displayed does not allow me to make use cases based on Unique values. For Example, here are some use cases -

1. One User Account used to login to multiple machines in a short span of time.
2. Single IP making multiple connections to other machines within the network via SMB/ telnet etc.
3. BruteForce Attempts

Please suggest if Watchers are not required for these Use Cases.

---

<div class="post-metadata">

**Author:** ![borna\_talebi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/borna_talebi/32/76317_2.png) [@borna\_talebi](https://discuss.elastic.co/u/borna_talebi)\
**Post date:** [September 26, 2020, 11:12am UTC](https://discuss.elastic.co/t/calling-alerts-from-watchers-to-detection-signals/250032/5 "2020-09-26T11:12:57Z")

</div>

instead of `_index : "alerts*"` try something like `agent.name : *` and see if it works.

> [@Suro](#):
>
> How do I check the mapping of the index.

you can also use [mapping API](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-get-mapping.html) for that.

---

<div class="post-metadata">

**Author:** ![Suro](https://avatars.discourse-cdn.com/v4/letter/s/b9bd4f/32.png) [@Suro](https://discuss.elastic.co/u/Suro)\
**Post date:** [September 26, 2020, 5:10pm UTC](https://discuss.elastic.co/t/calling-alerts-from-watchers-to-detection-signals/250032/6 "2020-09-26T17:10:28Z")

</div>

Tried that ....did not work -

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/d/4/d4b6a240a203699a3385e42e8d6cc3a0cc6ca45f.png)

---

<div class="post-metadata">

**Author:** ![Suro](https://avatars.discourse-cdn.com/v4/letter/s/b9bd4f/32.png) [@Suro](https://discuss.elastic.co/u/Suro)\
**Post date:** [September 26, 2020, 5:15pm UTC](https://discuss.elastic.co/t/calling-alerts-from-watchers-to-detection-signals/250032/7 "2020-09-26T17:15:17Z")

</div>

my bad the field name Agent doesnot exists in Alerts index. So I am trying one Use case specifically and trying to see if that is generated!!!!

---

<div class="post-metadata">

**Author:** ![Suro](https://avatars.discourse-cdn.com/v4/letter/s/b9bd4f/32.png) [@Suro](https://discuss.elastic.co/u/Suro)\
**Post date:** [September 26, 2020, 5:44pm UTC](https://discuss.elastic.co/t/calling-alerts-from-watchers-to-detection-signals/250032/8 "2020-09-26T17:44:05Z")

</div>

That did not work-

here is the discover query-

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/b/2/b22b86961e4d4cc91e325c95ba9e33319d3232b5.png)

With the same parameters, here is the Detection rule -

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/c/8/c88d52acb1293034ff8b992aba1503d7f456008e.png)

Still no detections.

---

<div class="post-metadata">

**Author:** ![Frank\_Hassanabad](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/frank_hassanabad/32/49255_2.png) [@Frank\_Hassanabad](https://discuss.elastic.co/u/Frank_Hassanabad)\
**Post date:** [September 28, 2020, 1:01pm UTC](https://discuss.elastic.co/t/calling-alerts-from-watchers-to-detection-signals/250032/9 "2020-09-28T13:01:30Z")

</div>

What's the mapping? Run something like this below in your dev tools and see what it is. The signals index mapping is that of ECS, so if there is a conflict or issue where the source index which is your alerts in this case is too far away from ECS then it more than likely is not going to work out for you.

In dev tools you can do:

```auto
GET /alerts*/_mapping

```

And then give us the output and we can see.

---

<div class="post-metadata">

**Author:** ![Suro](https://avatars.discourse-cdn.com/v4/letter/s/b9bd4f/32.png) [@Suro](https://discuss.elastic.co/u/Suro)\
**Post date:** [September 28, 2020, 1:15pm UTC](https://discuss.elastic.co/t/calling-alerts-from-watchers-to-detection-signals/250032/10 "2020-09-28T13:15:06Z")

</div>

Here is the output -

```auto
{
  "alerts" : {
    "mappings" : {
      "properties" : {
        "@timestamp" : {
          "type" : "date"
        },
        "agent" : {
          "properties" : {
            "hostname" : {
              "type" : "keyword"
            },
            "name" : {
              "type" : "keyword"
            },
            "version" : {
              "type" : "keyword"
            }
          }
        },
        "alert_id" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "alert_instance_id" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "alert_name" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "beat" : {
          "properties" : {
            "hostname" : {
              "type" : "keyword"
            },
            "name" : {
              "type" : "keyword"
            },
            "version" : {
              "type" : "keyword"
            }
          }
        },
        "ecs" : {
          "properties" : {
            "version" : {
              "type" : "keyword"
            }
          }
        },
        "faied_count_alert" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "host" : {
          "properties" : {
            "architecture" : {
              "type" : "keyword"
            },
            "containerized" : {
              "type" : "boolean"
            },
            "geo" : {
              "properties" : {
                "city_name" : {
                  "type" : "keyword"
                },
                "continent_name" : {
                  "type" : "keyword"
                },
                "country_iso_code" : {
                  "type" : "keyword"
                },
                "country_name" : {
                  "type" : "keyword"
                },
                "location" : {
                  "type" : "geo_point"
                },
                "name" : {
                  "type" : "keyword"
                },
                "region_iso_code" : {
                  "type" : "keyword"
                },
                "region_name" : {
                  "type" : "keyword"
                }
              }
            },
            "hostname" : {
              "type" : "keyword"
            },
            "id" : {
              "type" : "keyword"
            },
            "ip" : {
              "type" : "ip"
            },
            "mac" : {
              "type" : "keyword"
            },
            "name" : {
              "type" : "keyword"
            },
            "os" : {
              "properties" : {
                "build" : {
                  "type" : "keyword"
                },
                "codename" : {
                  "type" : "keyword"
                },
                "family" : {
                  "type" : "keyword"
                },
                "full" : {
                  "type" : "keyword"
                },
                "kernel" : {
                  "type" : "keyword"
                },
                "name" : {
                  "type" : "keyword"
                },
                "platform" : {
                  "type" : "keyword"
                },
                "version" : {
                  "type" : "keyword"
                }
              }
            },
            "type" : {
              "type" : "keyword"
            },
            "user" : {
              "properties" : {
                "email" : {
                  "type" : "keyword"
                },
                "full_name" : {
                  "type" : "keyword"
                },
                "group" : {
                  "properties" : {
                    "id" : {
                      "type" : "keyword"
                    },
                    "name" : {
                      "type" : "keyword"
                    }
                  }
                },
                "hash" : {
                  "type" : "keyword"
                },
                "id" : {
                  "type" : "keyword"
                },
                "name" : {
                  "type" : "keyword"
                }
              }
            }
          }
        },
        "host_count" : {
          "type" : "long"
        },
        "host_name" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "log" : {
          "properties" : {
            "file" : {
              "properties" : {
                "path" : {
                  "type" : "keyword"
                }
              }
            },
            "flags" : {
              "type" : "keyword"
            },
            "level" : {
              "type" : "keyword"
            },
            "offset" : {
              "type" : "long"
            },
            "source" : {
              "properties" : {
                "address" : {
                  "type" : "keyword"
                }
              }
            }
          }
        },
        "message" : {
          "type" : "text"
        },
        "severity" : {
          "type" : "long"
        },
        "source_host" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "source_host_alert" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "source_ip" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "source_user" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "source_user_alert" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "total_count" : {
          "type" : "long"
        },
        "triggered" : {
          "type" : "date"
        },
        "user_name" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "watch_id" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        }
      }
    }
  }
}

```

---

<div class="post-metadata">

**Author:** ![Frank\_Hassanabad](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/frank_hassanabad/32/49255_2.png) [@Frank\_Hassanabad](https://discuss.elastic.co/u/Frank_Hassanabad)\
**Post date:** [September 29, 2020, 5:27pm UTC](https://discuss.elastic.co/t/calling-alerts-from-watchers-to-detection-signals/250032/11 "2020-09-29T17:27:09Z")

</div>

Don't see any mapping conflicts with it at the moment. If you have access to your Kibana logs do you see any errors when the signals are being run?

---

<div class="post-metadata">

**Author:** ![Suro](https://avatars.discourse-cdn.com/v4/letter/s/b9bd4f/32.png) [@Suro](https://discuss.elastic.co/u/Suro)\
**Post date:** [September 30, 2020, 1:23pm UTC](https://discuss.elastic.co/t/calling-alerts-from-watchers-to-detection-signals/250032/12 "2020-09-30T13:23:28Z")

</div>

No Errors -

```auto
message:

alert executed: siem.signals:0eeb4d4c-2ed9-4098-91ca-6ae1a72d3720: 'Watcher Alerts'

 

event.provider:

alerting

 

event.action:

execute

 

event.start:

Sep 30, 2020 @ 12:56:29.207

 

event.end:

Sep 30, 2020 @ 12:56:31.056

 

event.duration:

1,849,000,000

 

event.outcome:

success

 

kibana.saved_objects:

{ "rel": "primary", "type": "alert", "id": "0eeb4d4c-2ed9-4098-91ca-6ae1a72d3720" }

 

kibana.server_uuid:

db28e422-3b70-4b7d-afbd-1c94869e3914

 

@timestamp:

Sep 30, 2020 @ 12:56:31.056

 

ecs.version:

1.5.0

 

_id:

bZgV33QBf6VWBkBR8PlR

 

_type:

_doc

 

_index:

.kibana-event-log-7.9.0-000002

 

_score:

 -

```

---

<div class="post-metadata">

**Author:** ![Frank\_Hassanabad](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/frank_hassanabad/32/49255_2.png) [@Frank\_Hassanabad](https://discuss.elastic.co/u/Frank_Hassanabad)\
**Post date:** [September 30, 2020, 1:59pm UTC](https://discuss.elastic.co/t/calling-alerts-from-watchers-to-detection-signals/250032/13 "2020-09-30T13:59:10Z")

</div>

You can put Kibana into a type of debug mode like so in kibana.yml to get more log output to help out:

```auto
logging.verbose: true
logging.events:
  {
    log: ['securitySolution', 'alerting', info', 'warning', 'error', 'fatal'],
    request: ['info', 'warning', 'error', 'fatal'],
    error: '*',
    ops: __no-ops__ ,
  }

```

---

<div class="post-metadata">

**Author:** ![Suro](https://avatars.discourse-cdn.com/v4/letter/s/b9bd4f/32.png) [@Suro](https://discuss.elastic.co/u/Suro)\
**Post date:** [September 30, 2020, 10:11pm UTC](https://discuss.elastic.co/t/calling-alerts-from-watchers-to-detection-signals/250032/14 "2020-09-30T22:11:50Z")

</div>

Interestingly I am seeing the same issue with another index related to defender events. Though I will try to get the yml file updated, did u see anyone else complaining about this.

---

<div class="post-metadata">

**Author:** ![Suro](https://avatars.discourse-cdn.com/v4/letter/s/b9bd4f/32.png) [@Suro](https://discuss.elastic.co/u/Suro)\
**Post date:** [October 1, 2020, 8:36am UTC](https://discuss.elastic.co/t/calling-alerts-from-watchers-to-detection-signals/250032/15 "2020-10-01T08:36:40Z")

</div>

Still getting the same. The setting suggested was performed but the logs still says success but no alert -

```auto
{
  "_index": ".kibana-event-log-7.9.0-000002",
  "_type": "_doc",
  "_id": "HXhF43QBf6VWBkBR6f7K",
  "_score": 1,
  "_source": {
    "event": {
      "provider": "alerting",
      "action": "execute",
      "start": "2020-10-01T08:27:22.474Z",
      "end": "2020-10-01T08:27:23.978Z",
      "duration": 1504000000,
      "outcome": "success"
    },
    "kibana": {
      "saved_objects": [
        {
          "rel": "primary",
          "type": "alert",
          "id": "0eeb4d4c-2ed9-4098-91ca-6ae1a72d3720"
        }
      ],
      "server_uuid": "db28e422-3b70-4b7d-afbd-1c94869e3914"
    },
    "message": "alert executed: siem.signals:0eeb4d4c-2ed9-4098-91ca-6ae1a72d3720: 'Watcher Alerts'",
    "@timestamp": "2020-10-01T08:27:23.978Z",
    "ecs": {
      "version": "1.5.0"
    }
  },
  "fields": {
    "event.start": [
      "2020-10-01T08:27:22.474Z"
    ],
    "event.end": [
      "2020-10-01T08:27:23.978Z"
    ],
    "@timestamp": [
      "2020-10-01T08:27:23.978Z"
    ]
  }
}

```

Similar problem with another index - windowsdefender

```auto
{
  "_index": ".kibana-event-log-7.9.0-000002",
  "_type": "_doc",
  "_id": "L4ZL43QBf6VWBkBRV1S0",
  "_score": 1,
  "_source": {
    "event": {
      "provider": "alerting",
      "action": "execute",
      "start": "2020-10-01T08:33:17.647Z",
      "end": "2020-10-01T08:33:19.796Z",
      "duration": 2149000000,
      "outcome": "success"
    },
    "kibana": {
      "saved_objects": [
        {
          "rel": "primary",
          "type": "alert",
          "id": "caa921df-65f1-492c-9923-b24ef5271dc3"
        }
      ],
      "server_uuid": "db28e422-3b70-4b7d-afbd-1c94869e3914"
    },
    "message": "alert executed: siem.signals:caa921df-65f1-492c-9923-b24ef5271dc3: 'Virus Infected Machines (Quarantine Failure) - SNOW'",
    "@timestamp": "2020-10-01T08:33:19.796Z",
    "ecs": {
      "version": "1.5.0"
    }
  },
  "fields": {
    "event.start": [
      "2020-10-01T08:33:17.647Z"
    ],
    "event.end": [
      "2020-10-01T08:33:19.796Z"
    ],
    "@timestamp": [
      "2020-10-01T08:33:19.796Z"
    ]
  }
}

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 29, 2020, 8:36am UTC](https://discuss.elastic.co/t/calling-alerts-from-watchers-to-detection-signals/250032/16 "2020-10-29T08:36:44Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.