# ML partition by two fields

**URL:** <https://discuss.elastic.co/t/ml-partition-by-two-fields/281831>\
**Category:** Elasticsearch\
**Tags:** elastic-stack-machine-learning, painless\
**Created:** [August 18, 2021, 2:15pm UTC](https://discuss.elastic.co/t/ml-partition-by-two-fields/281831 "2021-08-18T14:15:34Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![OmarDacca](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/omardacca/32/68171_2.png) [@OmarDacca](https://discuss.elastic.co/u/OmarDacca)\
**Post date:** [August 18, 2021, 2:15pm UTC](https://discuss.elastic.co/t/ml-partition-by-two-fields/281831/1 "2021-08-18T14:15:35Z")

</div>

Hi,

ES v 7.13.2

I'm trying to create the following ML job and not sure if I'm doing it correctly.

indices: user\_events\*  
every document looks something like:

```auto
{
    "date_time": "2021-07-16T01:00:00.000Z",
    "clientId": 123,
    "domainsGroup": "OrgName1",
    "event_key": "add_to_cart",
    "event": "Add to cart",
    "ok": false/true
}

```

I want to detect if there are an increase or decrease in the number of failed goals (ok=false) - (per domainsGroup and event). In other words, I want to split the data by domainsGroup\_event and then count per hour, how many i.e. failed Add to carts OrgName1 had - if there was a peak in an hour, I want to do some actions..

I tried in many ways, nothing worked, but specificly, can't understand why the following doesn't work.. ?  
note: every document must have all the field mentioned above, and event, event\_key, domainsGroup and clientId are all keywords.

running this job gives me this error: **"Datafeed is encountering errors extracting data: runtime error"**

```auto
PUT _ml/anomaly_detectors/hourly_goal_failures_anomalies_job_v1
{
  "description" : "goal failures anomalies job",
  "analysis_config": {
    "bucket_span": "1h",
    "detectors": [
      {
        "function": "count",
        "partition_field_name": "domainsGroup_event",
        "detector_description": "goal failures count"
      }
    ],
    "influencers": [
      "domainsGroup",
      "clientId",
      "event"
    ]
  },
  "analysis_limits": {
    "model_memory_limit": "500MB"
  },
  "data_description": {
    "time_field": "date_time",
    "time_format": "epoch_ms"
  },
  "model_snapshot_retention_days": 10,
  "daily_model_snapshot_retention_after_days": 1,
  "results_index_name": "",
  "allow_lazy_open": false,
  "groups": []
}

PUT _ml/datafeeds/datafeed-hourly_goal_failures_anomalies_job_v1
{
  "query_delay": "5m",
  "query": {
    "bool": {
      "filter": [
        {
          "term": {
            "ok": false
          }
        }
      ]
    }
  },
  "indices": [
    "user_events*"
  ],
  "script_fields": {
    "domainsGroup_event": {
      "script": {
        "source": "doc['domainsGroup'].value + doc['event'].value",
        "lang": "painless"
      },
      "ignore_failure": true
    }
  },
  "scroll_size": 1000,
  "delayed_data_check_config": {
    "enabled": true
  },
  "job_id": "hourly_goal_failures_anomalies_job_v1",
  "datafeed_id": "datafeed-hourly_goal_failures_anomalies_job_v1"
}

```

---

<div class="post-metadata">

**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\
**Post date:** [August 18, 2021, 4:31pm UTC](https://discuss.elastic.co/t/ml-partition-by-two-fields/281831/2 "2021-08-18T16:31:06Z")

</div>

What do you see if you run datafeed preview?

```auto
GET _ml/datafeeds/datafeed-hourly_goal_failures_anomalies_job_v1/_preview

```

What's the mappings for your index?

```auto
GET user_events*/_mapping

```

---

<div class="post-metadata">

**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\
**Post date:** [August 18, 2021, 4:32pm UTC](https://discuss.elastic.co/t/ml-partition-by-two-fields/281831/3 "2021-08-18T16:32:14Z")

</div>

You should also know you can get a double split naturally by using both the `partition_field_name` and the `by_field_name` - no need to artificially create a `script_field`

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 15, 2021, 4:32pm UTC](https://discuss.elastic.co/t/ml-partition-by-two-fields/281831/4 "2021-09-15T16:32:32Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.