# Anomaly detection / ML Setup

**URL:** <https://discuss.elastic.co/t/anomaly-detection-ml-setup/272378>\
**Category:** Logs\
**Tags:** elastic-stack-machine-learning\
**Created:** [May 7, 2021, 7:31am UTC](https://discuss.elastic.co/t/anomaly-detection-ml-setup/272378 "2021-05-07T07:31:04Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![tsp](https://avatars.discourse-cdn.com/v4/letter/t/76d3ee/32.png) [@tsp](https://discuss.elastic.co/u/tsp)\
**Post date:** [May 7, 2021, 7:31am UTC](https://discuss.elastic.co/t/anomaly-detection-ml-setup/272378/1 "2021-05-07T07:31:04Z")

</div>

When setting up the ML within the Anomaly detection (Stack 11.1.2) , it says the following:

At least one index matching logs-pattern-\* has a field called event.dataset without the correct type.

Here is a sample record:

```auto
{
  "_index": "cen-logs-2021.05.07",
  "_type": "_doc",
  "_id": "y-mkRXkBxSJkcJbaDJAK",
  "_version": 1,
  "_score": null,
  "_source": {
    "time": "2021-05-07T07:04:08.45951746Z",
    "@version": "1",
    "host": {
      "name": "XXXX.YYYY.ZZZZ.net",
      "hostname": "XXXX.YYYY.ZZZZ.net"
    },
    "@timestamp_fluentbit": 1620371048.459517,
    "ecs": {
      "version": "1.9.0"
    },
    "topickey": "a_topic",
    "message": "2021-05-07T07:04:08.459Z\tERROR\t[kafka]\tkafka/client.go:317\tKafka (topic=a_topic): kafka: client has run out of available brokers to talk to (Is your cluster reachable?)\n",
    "clustername": AAAA.XXXX.YYYY.ZZZZ.net",
    "@timestamp": "2021-05-07T07:04:09.304Z",
    "kubernetes": {
      "pod_id": "7e925054-a7df-11eb-b927-94f128a205f0",
      "container_name": "heartbeat",
      "host": "XXXX.YYYY.ZZZZ.net",
      "container_hash": "BBBB-container-registry.docker.YYYY.net/elastic/heartbeat@sha256:14a150d5c4430076477860715faf0fb63e0c838e021c3dfd28ddda6710c043de",
      "namespace_name": "XXXX-acc",
      "pod_name": "heartbeat-deployment-f67d8d945-qgjqz",
      "docker_id": "31d306622560865f2cb6d5f628971f27d3e58ab597e59e3319504cd98909796f",
      "container_image": "BBBB-container-registry.docker.XXXX.net/elastic/heartbeat@sha256:14a150d5c4430076477860715faf0fb63e0c838e021c3dfd28ddda6710c043de",
      "pod": {
        "id": "7e925054-a7df-11eb-b927-94f128a205f0"
      },
      "labels": {
        "release": "RELEASE-NAME",
        "pod-template-hash": "923848501",
        "chart": "heartbeat-7.11.2",
        "app_": "RELEASE-NAME-heartbeat-metrics"
      },
      "annotations": {
        "configChecksum": "19b27e247c38d2781ca94daa41c6cf6e146dcec527004438e7280e9b5ee65c4",
        "openshift.io/scc": "restricted",
        "loggingkafkatopic": "a_topc"
      }
    },
    "stream": "stderr",
    "log": {
      "file": {
        "path": "/Container/log"
      }
    },
    "container": {
      "id": "AAAA-container-registry.docker.XXXX.net/elastic/heartbeat@sha256:14a150d5c4430076477860715faf0fb63e0c838e021c3dfd28ddda6710c043de"
    },
    "event": {
      "kind": "event",
      "type": "info",
      "category": "host",
      "dataset": "module.dataset"
    }
  },
  "fields": {
    "@timestamp": [
      "2021-05-07T07:04:09.304Z"
    ],
    "time": [
      "2021-05-07T07:04:08.459Z"
    ]
  },
  "sort": [
    1620371049304
  ]
}

```

Any hint appreciated 🙂

Thanks

---

<div class="post-metadata">

**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\
**Post date:** [May 9, 2021, 5:55pm UTC](https://discuss.elastic.co/t/anomaly-detection-ml-setup/272378/2 "2021-05-09T17:55:38Z")

</div>

> [@tsp](#):
>
> ```auto
> "_index": "cen-logs-2021.05.07",
> 
> ```

Are you enabling ML from the Logs UI?

Judging by your error message and your sample document, it looks like your index of `cen-logs-2021.05.07` doesn't match the default index patterns in the Logs UI (or you may have logs in some other indices that _do_ match the pattern, but don't conform to having the required `event.dataset` field).

---

<div class="post-metadata">

**Author:** ![tsp](https://avatars.discourse-cdn.com/v4/letter/t/76d3ee/32.png) [@tsp](https://discuss.elastic.co/u/tsp)\
**Post date:** [May 11, 2021, 7:14am UTC](https://discuss.elastic.co/t/anomaly-detection-ml-setup/272378/3 "2021-05-11T07:14:12Z")

</div>

Thanks, I was indeed enabling it through the UI.

I found out afterwards that some data was shipped into a complete different format, now that both formats are handled, it works as a charm. 🙂  
Thanks.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 8, 2021, 7:14am UTC](https://discuss.elastic.co/t/anomaly-detection-ml-setup/272378/4 "2021-06-08T07:14:46Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.