# ML multi metric split filed only have keywork

**URL:** <https://discuss.elastic.co/t/ml-multi-metric-split-filed-only-have-keywork/280888>\
**Category:** Kibana\
**Tags:** elastic-stack-machine-learning\
**Created:** [August 10, 2021, 7:46am UTC](https://discuss.elastic.co/t/ml-multi-metric-split-filed-only-have-keywork/280888 "2021-08-10T07:46:04Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![quyennguyen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/quyennguyen/32/91626_2.png) [@quyennguyen](https://discuss.elastic.co/u/quyennguyen)\
**Post date:** [August 10, 2021, 7:46am UTC](https://discuss.elastic.co/t/ml-multi-metric-split-filed-only-have-keywork/280888/1 "2021-08-10T07:46:04Z")

</div>

I'm just a beginner of ML so I didn't know why this happened in my ELK.  
I want to create new job in multi metric so I following those step  
Pick "Fail login" search index  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/7/4/74c45952ee149435dd10c3107aafb3cd481aef47.png)  
Then pick multi metric -\> time range "use full filebeat\* data"

Pick file "high-count(event rate)"  
But when I want to pick split field and influencersit they only have field.keyword  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/b/f/bf067e9834b68eb92e975caf1a3c03d77eac785c.png)

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/8/c/8c97ca4ceaf30c932a09cff63d5821008dfb623d.png)

So when I create metric it has error

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/3/e/3edfeb6d21958b6687765d6a3e1ae0211fa944d6.png)

How can I fix this?  
I know this happened because I used field.keywork, but I didn't have other choice.  
What's happen with my log?

---

<div class="post-metadata">

**Author:** ![BenTrent](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bentrent/32/33915_2.png) [@BenTrent](https://discuss.elastic.co/u/BenTrent)\
**Post date:** [August 10, 2021, 11:36am UTC](https://discuss.elastic.co/t/ml-multi-metric-split-filed-only-have-keywork/280888/2 "2021-08-10T11:36:24Z")

</div>

@quyennguyen, there is an advanced option to give a job its own index.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/f/8/f82d099dd63317ae7694c273d7de0d18c9947c08.png)

By default jobs share an index to reduce the overall cluster shard count. But, for situations like this (or for very large jobs), selecting "Use dedicated index" is a good idea.

---

<div class="post-metadata">

**Author:** ![quyennguyen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/quyennguyen/32/91626_2.png) [@quyennguyen](https://discuss.elastic.co/u/quyennguyen)\
**Post date:** [August 10, 2021, 2:25pm UTC](https://discuss.elastic.co/t/ml-multi-metric-split-filed-only-have-keywork/280888/3 "2021-08-10T14:25:35Z")

</div>

It worked somehow, but seem like I didn't get the result

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/f/e/fe6e0bcf00ae1bb0c7469e237a525f054aa40031.png)

Do you know why I don't have fields but only have fields.keywork?

---

<div class="post-metadata">

**Author:** ![BenTrent](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bentrent/32/33915_2.png) [@BenTrent](https://discuss.elastic.co/u/BenTrent)\
**Post date:** [August 10, 2021, 2:46pm UTC](https://discuss.elastic.co/t/ml-multi-metric-split-filed-only-have-keywork/280888/4 "2021-08-10T14:46:46Z")

</div>

@quyennguyen , I don't understand your question. Are you wondering why there are no influencer results?

If thats the case, could you provide the job and datafeed configurations (with sensitive information removed, if any exists in the configuration).

---

<div class="post-metadata">

**Author:** ![quyennguyen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/quyennguyen/32/91626_2.png) [@quyennguyen](https://discuss.elastic.co/u/quyennguyen)\
**Post date:** [August 10, 2021, 3:13pm UTC](https://discuss.elastic.co/t/ml-multi-metric-split-filed-only-have-keywork/280888/5 "2021-08-10T15:13:49Z")

</div>

I'm wondering why in split field on have fields.keywork but not fields. Because when I did it the first time, it has system.auth.hostname but not system.auth.hostname.keywork, and I get the result I want. But I deleted it for some reason.

About job and datafeed config, I don't know where to get it, 'cause I really empty in this side. Can you show me how to get it send to you?

---

<div class="post-metadata">

**Author:** ![BenTrent](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bentrent/32/33915_2.png) [@BenTrent](https://discuss.elastic.co/u/BenTrent)\
**Post date:** [August 10, 2021, 6:21pm UTC](https://discuss.elastic.co/t/ml-multi-metric-split-filed-only-have-keywork/280888/6 "2021-08-10T18:21:30Z")

</div>

> [@quyennguyen](#):
>
> fields.keywork

It has `keyword` because you specified those values. `keyword` signifies a keyword field type. It means your text field is indexed as a keyword as well as a text type.

I am guessing it worked before because your data has somehow changed. Possibly not all the indices have they `keyword` field, or that field was added after data was already indexed.

To get the configs, you should be able to get them from the machine learning job management UI: [Anomaly detection | Kibana Guide [8.11] | Elastic](https://www.elastic.co/guide/en/kibana/current/xpack-ml-anomalies.html)

Select the `JSON` tab.

---

<div class="post-metadata">

**Author:** ![quyennguyen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/quyennguyen/32/91626_2.png) [@quyennguyen](https://discuss.elastic.co/u/quyennguyen)\
**Post date:** [August 10, 2021, 7:07pm UTC](https://discuss.elastic.co/t/ml-multi-metric-split-filed-only-have-keywork/280888/7 "2021-08-10T19:07:54Z")

</div>

OK. After created job with keyword field, I got this

```auto
{
  "job_id": "login_ssh_multi_metric",
  "job_type": "anomaly_detector",
  "job_version": "7.13.3",
  "create_time": 1628622270365,
  "finished_time": 1628622303319,
  "model_snapshot_id": "1628622302",
  "description": "",
  "analysis_config": {
    "bucket_span": "1m",
    "detectors": [
      {
        "detector_description": "high_count partitionfield=\"system.auth.hostname.keyword\"",
        "function": "high_count",
        "partition_field_name": "system.auth.hostname.keyword",
        "detector_index": 0
      }
    ],
    "influencers": [
      "system.auth.hostname.keyword",
      "system.auth.ssh.ip.keyword"
    ]
  },
  "analysis_limits": {
    "model_memory_limit": "11mb",
    "categorization_examples_limit": 4
  },
  "data_description": {
    "time_field": "@timestamp",
    "time_format": "epoch_ms"
  },
  "model_plot_config": {
    "enabled": false,
    "annotations_enabled": false
  },
  "model_snapshot_retention_days": 10,
  "daily_model_snapshot_retention_after_days": 1,
  "results_index_name": "custom-login_ssh_multi_metric",
  "allow_lazy_open": false,
  "data_counts": {
    "job_id": "login_ssh_multi_metric",
    "processed_record_count": 1685,
    "processed_field_count": 958,
    "input_bytes": 90377,
    "input_field_count": 958,
    "invalid_date_count": 0,
    "missing_field_count": 2412,
    "out_of_order_timestamp_count": 0,
    "empty_bucket_count": 32938,
    "sparse_bucket_count": 2,
    "bucket_count": 32968,
    "earliest_record_timestamp": 1626604733000,
    "latest_record_timestamp": 1628582785000,
    "last_data_time": 1628622272689,
    "latest_empty_bucket_timestamp": 1628582640000,
    "latest_sparse_bucket_timestamp": 1628529060000,
    "input_record_count": 1685,
    "log_time": 1628622272689,
    "latest_bucket_timestamp": 1628582760000
  },
  "model_size_stats": {
    "job_id": "login_ssh_multi_metric",
    "result_type": "model_size_stats",
    "model_bytes": 82740,
    "peak_model_bytes": 104736,
    "model_bytes_exceeded": 0,
    "model_bytes_memory_limit": 11534336,
    "total_by_field_count": 4,
    "total_over_field_count": 0,
    "total_partition_field_count": 3,
    "bucket_allocation_failures_count": 0,
    "memory_status": "ok",
    "assignment_memory_basis": "current_model_bytes",
    "categorized_doc_count": 0,
    "total_category_count": 0,
    "frequent_category_count": 0,
    "rare_category_count": 0,
    "dead_category_count": 0,
    "failed_category_count": 0,
    "categorization_status": "ok",
    "log_time": 1628622302104,
    "timestamp": 1628582700000
  },
  "forecasts_stats": {
    "total": 0,
    "forecasted_jobs": 0
  },
  "state": "closed",
  "timing_stats": {
    "job_id": "login_ssh_multi_metric",
    "bucket_count": 32968,
    "total_bucket_processing_time_ms": 18977.00000000005,
    "minimum_bucket_processing_time_ms": 0,
    "maximum_bucket_processing_time_ms": 532,
    "average_bucket_processing_time_ms": 0.5756187818490673,
    "exponential_average_bucket_processing_time_ms": 1.1737351400660319,
    "exponential_average_bucket_processing_time_per_hour_ms": 60.74121520456408
  },
  "datafeed_config": {
    "datafeed_id": "datafeed-login_ssh_multi_metric",
    "job_id": "login_ssh_multi_metric",
    "query_delay": "90505ms",
    "chunking_config": {
      "mode": "auto"
    },
    "indices_options": {
      "expand_wildcards": [
        "open"
      ],
      "ignore_unavailable": false,
      "allow_no_indices": true,
      "ignore_throttled": true
    },
    "query": {
      "bool": {
        "must": [
          {
            "match_all": {}
          }
        ],
        "filter": [
          {
            "match_phrase": {
              "system.auth.ssh.event": "Failed password"
            }
          }
        ],
        "must_not": []
      }
    },
    "indices": [
      "filebeat*"
    ],
    "scroll_size": 1000,
    "delayed_data_check_config": {
      "enabled": true
    },
    "state": "stopped",
    "timing_stats": {
      "job_id": "login_ssh_multi_metric",
      "search_count": 4,
      "bucket_count": 32968,
      "total_search_time_ms": 29,
      "average_search_time_per_bucket_ms": 0.0008796408638679932,
      "exponential_average_search_time_per_hour_ms": 20.478633887048332
    }
  }
}

```

The result didn't have value and system. auth.ssh.ip.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/a/7/a7bff23c303820240790afbb40f337356a21fa0a.png)

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/6/9/6903700a204ce6ecd16a40392d3715a9ecdedffe.png)

---

<div class="post-metadata">

**Author:** ![BenTrent](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bentrent/32/33915_2.png) [@BenTrent](https://discuss.elastic.co/u/BenTrent)\
**Post date:** [August 10, 2021, 7:34pm UTC](https://discuss.elastic.co/t/ml-multi-metric-split-filed-only-have-keywork/280888/8 "2021-08-10T19:34:46Z")

</div>

The document values for `system.auth.hostname.keyword` and `system.auth.ssh.ip.keyword` where anomalies are occurring seem to be `""`

I have just confirmed with my own data that two jobs, one looking at the text field and one looking at the `keyword` field resulted in the same anomalies and influencers. This is only the case if all your data has the `keyword` field.

---

<div class="post-metadata">

**Author:** ![quyennguyen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/quyennguyen/32/91626_2.png) [@quyennguyen](https://discuss.elastic.co/u/quyennguyen)\
**Post date:** [August 10, 2021, 8:18pm UTC](https://discuss.elastic.co/t/ml-multi-metric-split-filed-only-have-keywork/280888/9 "2021-08-10T20:18:37Z")

</div>

I still can't get it, my English maybe not good enough to get your idea.

Do you mean this error happen because all my log has the keyword field?

What can I do to fix this?

---

<div class="post-metadata">

**Author:** ![quyennguyen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/quyennguyen/32/91626_2.png) [@quyennguyen](https://discuss.elastic.co/u/quyennguyen)\
**Post date:** [August 11, 2021, 8:12am UTC](https://discuss.elastic.co/t/ml-multi-metric-split-filed-only-have-keywork/280888/10 "2021-08-11T08:12:41Z")

</div>

I just delete log in a few days and resolve the problem, but then it got some error with datafeed, do you know why this happened?

 ![Capture](https://us1.discourse-cdn.com/elastic/original/3X/f/c/fc2fa3b8893137b3aa8e96de9a570b8ab4214f08.png)

---

<div class="post-metadata">

**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\
**Post date:** [August 11, 2021, 11:30am UTC](https://discuss.elastic.co/t/ml-multi-metric-split-filed-only-have-keywork/280888/11 "2021-08-11T11:30:02Z")

</div>

That problem occurs when the datafeed cannot successfully execute a search against the index where the raw data to be analyzed resides. You might have a problem with the health of elasticsearch itself. Look in the elasticsearch.log for errors. If you need help with core elasticsearch, there's a section in this forum for that. See [Elasticsearch - Discuss the Elastic Stack](https://discuss.elastic.co/c/elastic-stack/elasticsearch/)

---

<div class="post-metadata">

**Author:** ![rcowart](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rcowart/32/88091_2.png) [@rcowart](https://discuss.elastic.co/u/rcowart)\
**Post date:** [August 11, 2021, 2:30pm UTC](https://discuss.elastic.co/t/ml-multi-metric-split-filed-only-have-keywork/280888/12 "2021-08-11T14:30:44Z")

</div>

Let's focus on the underlying issue.

The split feature works on "aggregatable" fields. These include things like integers, IP addresses and a few others. For fields that hold strings they can be indexed two ways:

- as type `text`, which means they are _analyzed_ to support the free-text search features of Elasticsearch, but can NOT be used for aggregations
- as type `keyword` which means they are always evaluated for their absolute value, which they are "aggregatable" and thus support aggregation-based queries, but NOT free-text search.

If no index mappings are provided (usually in the form of an index template) which specifies the types of all fields in the index, the _default_ mappings will be used. The default mappings will index a string field as `text`, and index it a 2nd time as `keyword` appending `.keyword` to your field name.

NOTE: index mappings are also where you would specify fields containing IP addresses (which I also see in your data) as type `ip`.

So back to your original question... the reason that you only see the `.keyword` fields is that those are the only fields that are _aggregatable_. But this is likely not how you want it to work. Many of those fields do not make sense for free-text search and should be defined as keywords in the index mappings, without appending anything to the field names.

So what do you need to do? The answer is that you need to provide an index mapping, [ideally via an index template](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-templates.html), which correctly defines the data type of each field. Index templates can also be used to specify other index settings, such as the number of shards and replicas.

After your data is being indexed correctly, the ML features will work as you are expecting.

---

<div class="post-metadata">

**Author:** ![quyennguyen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/quyennguyen/32/91626_2.png) [@quyennguyen](https://discuss.elastic.co/u/quyennguyen)\
**Post date:** [August 13, 2021, 9:38am UTC](https://discuss.elastic.co/t/ml-multi-metric-split-filed-only-have-keywork/280888/13 "2021-08-13T09:38:37Z")

</div>

I tried to reinstall all clients I have with the same version of filebeat. And all fixed.

Thanks for helping me.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 10, 2021, 9:38am UTC](https://discuss.elastic.co/t/ml-multi-metric-split-filed-only-have-keywork/280888/14 "2021-09-10T09:38:45Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.