# "could not load fields from index" in datafeed while creating a ML job

**URL:** <https://discuss.elastic.co/t/could-not-load-fields-from-index-in-datafeed-while-creating-a-ml-job/220693>\
**Category:** Elasticsearch\
**Tags:** elastic-stack-machine-learning\
**Created:** [February 24, 2020, 4:13pm UTC](https://discuss.elastic.co/t/could-not-load-fields-from-index-in-datafeed-while-creating-a-ml-job/220693 "2020-02-24T16:13:00Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![higee](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/higee/32/22343_2.png) [@higee](https://discuss.elastic.co/u/higee)\
**Post date:** [February 24, 2020, 4:13pm UTC](https://discuss.elastic.co/t/could-not-load-fields-from-index-in-datafeed-while-creating-a-ml-job/220693/1 "2020-02-24T16:13:00Z")

</div>

Hi,

I am trying to create a ML job with _Advanced Job Creation._ The thing is I get an error from **datafeed** tab that it " **Could not load fields from index**". I wonder what exactly is going wrong. For instance, if I go to _Data Preview_ tab, I could see the fields in the indices. And also I was able to create a ML job without clicking **move to advanced job configuration**.

For your inforamation, I am using 7.4.2 for Elasticsearch, Logstash, Kibana. And the \_source field for these indices are all enabled.

Does anyone have an idea what I should do here? Oh yes, I will try creating a Job via Python client.

Have a look at the following error messages I got from Kibana.

![image](https://us1.discourse-cdn.com/elastic/original/3X/3/f/3f5add60da8d2f23a88b7c7c1ef7e421586ce5f9.png)  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/4/a/4a5dadc8c27164143e91f483d1669cf154e504ef.png)

Thanks,

---

<div class="post-metadata">

**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\
**Post date:** [February 24, 2020, 5:17pm UTC](https://discuss.elastic.co/t/could-not-load-fields-from-index-in-datafeed-while-creating-a-ml-job/220693/2 "2020-02-24T17:17:20Z")

</div>

I'd need to know more about:

1. What the actual detector configuration is (i.e. what fields do you expect to analyze in the ML job)
2. What the `mapping` of the index you want to analyze looks like and perhaps a sample document in that index

I don't want to jump to conclusions here, but explicitly setting the index to `test-2020-01.01` is probably NOT what you really want to do as this appears to be only 1 day's worth of data. You would normally create an index pattern (such as `test-*` that will match all timestamp-named indices and ML will handle the proper querying of the data in a chronological way over time)

---

<div class="post-metadata">

**Author:** ![higee](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/higee/32/22343_2.png) [@higee](https://discuss.elastic.co/u/higee)\
**Post date:** [February 24, 2020, 5:27pm UTC](https://discuss.elastic.co/t/could-not-load-fields-from-index-in-datafeed-while-creating-a-ml-job/220693/3 "2020-02-24T17:27:44Z")

</div>

Hi @richcollier,

Thanks for the reply. Let me go through one by one.

> 1. What the actual detector configuration is (i.e. what fields do you expect to analyze in the ML job)

```auto
"detectors": [
      {
        "function": "high_count",
        "detector_description": "high_count over request_header_forwarded_for",
        "over_field_name": "request_header_forwarded_for"
      }
    ]

```

> 1. What the `mapping` of the index you want to analyze looks like and perhaps a sample document in that index

Since I'm using templates for these indices, let me paste index template.

```auto
GET /_template/test
{
  "test" : {
    "order" : 0,
    "index_patterns" : [
      "test-*"
    ],
    "settings" : {
      "index" : {
        "number_of_shards" : "2"
      }
    },
    "mappings" : {
      "properties" : {
        "request_header_forwarded_for" : {
          "type" : "keyword"
        },
        "http_status_code" : {
          "type" : "keyword"
        },
        "request_header_user_agent" : {
          "type" : "keyword"
        },
        "http_verb" : {
          "type" : "keyword"
        }
      }
    },
    "aliases" : { }
  }
}

```

> I don't want to jump to conclusions here, but explicitly setting the index to `test-2020-01.01` is probably NOT what you really want to do as this appears to be only 1 day's worth of data. You would normally create an index pattern (such as `test-*` that will match all timestamp-named indices and ML will handle the proper querying of the data in a chronological way over time)

Oh yeah I'm not using `test-2020-01.01` in Kibana. I'm using proper index pattern as you mentioned. It was just a mistake while pasting here.

Hope this gives you more context on what I'm trying to do. Let me know if you need more information.

Thanks

---

<div class="post-metadata">

**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\
**Post date:** [February 24, 2020, 6:03pm UTC](https://discuss.elastic.co/t/could-not-load-fields-from-index-in-datafeed-while-creating-a-ml-job/220693/4 "2020-02-24T18:03:40Z")

</div>

Your config looks sensible.

Apparently that error can occur if the index is either empty or if the user who is configuring the job doesn't have permissions to read that index.

Validate that this is not the case and also what if anything do you see if you select the "Data Preview" tab? What errors, if any get logged to elasticsearch.log at that time?

---

<div class="post-metadata">

**Author:** ![higee](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/higee/32/22343_2.png) [@higee](https://discuss.elastic.co/u/higee)\
**Post date:** [February 25, 2020, 12:20pm UTC](https://discuss.elastic.co/t/could-not-load-fields-from-index-in-datafeed-while-creating-a-ml-job/220693/5 "2020-02-25T12:20:31Z")

</div>

To answer your questions one by one,

> Aparently that error can occur if the index is either empty

Indices in `test-*` index pattern are `test-2020.01.01`, `test-2020.01.02` and so on.

```auto
GET /test-*/_count?
{
  "count" : 8462636,
  "_shards" : {
    "total" : 14,
    "successful" : 14,
    "skipped" : 0,
    "failed" : 0
  }
}

```

> if the user who is configuring the job doesn't have permissions to read that index.

The user I am using has some roles attached to it.

- machine\_learning\_user
- machine\_learning\_admin
- kibana\_user
- log\_reader
- test (custom role we added)
  - Cluster privileges : monitor\_ml, manage\_index\_templates, manage
  - Run As privileges : None
  - Index privileges
    - indices : test-\*
    - privileges : all

> Validate that this is not the case

From the details above, I think I can conclude the indices are not empty and that I have right permissions.

> what if anything do you see if you select the "Data Preview" tab?

Yeah I see the documents with the fields I need. For your information, I had to scrub `id` and `request_header_forwareded_for` fields. They do contain legitimate values though.

```auto
[
  {
    "_index": "test-2020.02.05",
    "_type": "_doc",
    "_id": "",
    "_score": 2,
    "_source": {
      "request_header_forwarded_for": "",
      "@timestamp": "2020-02-05T00:27:38.381Z"
    }
  },
  {
    "_index": "test-2020.02.05",
    "_type": "_doc",
    "_id": "",
    "_score": 2,
    "_source": {
      "request_header_forwarded_for": "",
      "@timestamp": "2020-02-05T00:27:34.719Z"
    }
  }
⋮
]

```

> What errors, if any get logged to elasticsearch.log at that time?

What I got while reproducing the error is as follows.

```auto
Feb 25, 2020, 12:34:35 PM UTC
WARN
i2@eu-west-1c
[instance-0000000002] [GET /_xpack/ml/anomaly_detectors/_stats] is deprecated! Use [GET /_ml/anomaly_detectors/_stats] instead.
Feb 25, 2020, 12:34:35 PM UTC

WARN
i5@eu-west-1a
[instance-0000000005] [GET /_xpack/ml/anomaly_detectors] is deprecated! Use [GET /_ml/anomaly_detectors] instead.
Feb 25, 2020, 12:34:04 PM UTC

WARN
i1@eu-west-1c
[instance-0000000001] [GET /_xpack/ml/anomaly_detectors/_stats] is deprecated! Use [GET /_ml/anomaly_detectors/_stats] instead.
Feb 25, 2020, 12:34:04 PM UTC

WARN
i2@eu-west-1c
[instance-0000000002] [GET /_xpack/ml/info] is deprecated! Use [GET /_ml/info] instead.

```

---

<div class="post-metadata">

**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\
**Post date:** [February 25, 2020, 6:56pm UTC](https://discuss.elastic.co/t/could-not-load-fields-from-index-in-datafeed-while-creating-a-ml-job/220693/6 "2020-02-25T18:56:41Z")

</div>

Hmm...those are just informational messages in the log and shouldn't impede the operation.

I am a bit curious about the user's role. Have you, perhaps, tried to repeat what you're doing with a different user? Perhaps the `elastic` superuser? Just for comparison... Perhaps there is some permissions clash going on

---

<div class="post-metadata">

**Author:** ![higee](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/higee/32/22343_2.png) [@higee](https://discuss.elastic.co/u/higee)\
**Post date:** [February 26, 2020, 10:23am UTC](https://discuss.elastic.co/t/could-not-load-fields-from-index-in-datafeed-while-creating-a-ml-job/220693/7 "2020-02-26T10:23:46Z")

</div>

Unfortunately `superuser` didn't help. I will try creating a ML job with Python client to see if I can bypass this error. Please do let me know if you have any other idea.

Thanks

---

<div class="post-metadata">

**Author:** ![higee](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/higee/32/22343_2.png) [@higee](https://discuss.elastic.co/u/higee)\
**Post date:** [February 26, 2020, 2:28pm UTC](https://discuss.elastic.co/t/could-not-load-fields-from-index-in-datafeed-while-creating-a-ml-job/220693/8 "2020-02-26T14:28:17Z")

</div>

@richcollier I was able to bypass the error above by creating a ML job and plugging in datafeed via REST API calls. So this is no longer a blocker on my side. Still, I would appreciate if we can find a way to do it from Kibana UI.

Thanks,

Gee

---

<div class="post-metadata">

**Author:** ![James\_Gowdy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/james_gowdy/32/74175_2.png) [@James\_Gowdy](https://discuss.elastic.co/u/James_Gowdy)\
**Post date:** [February 26, 2020, 3:36pm UTC](https://discuss.elastic.co/t/could-not-load-fields-from-index-in-datafeed-while-creating-a-ml-job/220693/9 "2020-02-26T15:36:59Z")

</div>

The advanced wizard uses field caps to determine the fields.  
Would it be possible to call `GET /test-*/_field_caps` and paste the results here?  
Also, are there any errors in the browser console when using that page?

---

<div class="post-metadata">

**Author:** ![higee](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/higee/32/22343_2.png) [@higee](https://discuss.elastic.co/u/higee)\
**Post date:** [February 26, 2020, 7:42pm UTC](https://discuss.elastic.co/t/could-not-load-fields-from-index-in-datafeed-while-creating-a-ml-job/220693/10 "2020-02-26T19:42:50Z")

</div>

Hi @James_Gowdy, I'll answer your questions one by one.

> Would it be possible to call `GET /test-*/_field_caps` and paste the results here?

```auto
GET /test-*/_field_caps
{
  "error": {
    "root_cause": [
      {
        "type": "illegal_argument_exception",
        "reason": "specified fields can't be null or empty"
      }
    ],
    "type": "illegal_argument_exception",
    "reason": "specified fields can't be null or empty"
  },
  "status": 400
}

```

If I specify fields, however, I got valid response though.

```auto
GET /test-*/_field_caps?fields=request_header_forwarded_for,request_haproxy_acl,http_status_code,http_verb,request_header_user_agent
{
  "indices" : [
    "test-2020.01.01",
    "test-2020.01.02",
    "test-2020.01.03",
    "test-2020.01.04",
    "test-2020.01.05",
    "test-2020.01.06",
            ⋮
  ],
  "fields" : {
    "request_header_forwarded_for" : {
      "keyword" : {
        "type" : "keyword",
        "searchable" : true,
        "aggregatable" : true
      }
    },
    "http_status_code" : {
      "keyword" : {
        "type" : "keyword",
        "searchable" : true,
        "aggregatable" : true
      }
    },
    "request_header_user_agent" : {
      "keyword" : {
        "type" : "keyword",
        "searchable" : true,
        "aggregatable" : true
      }
    },
    "http_verb" : {
      "keyword" : {
        "type" : "keyword",
        "searchable" : true,
        "aggregatable" : true
      }
    }
  }
}

```

> Also, are there any errors in the browser console when using that page?

One thing I found was

```auto
Refused to execute inline script because it violates the following 
Content Security Policy directive: "script-src 'unsafe-eval' 'self'". 
Either the 'unsafe-inline' keyword, a hash ('sha256-...'), or a 
nonce ('nonce-...') is required to enable inline execution.

```

Hope this helps.

Thanks

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 25, 2020, 7:42pm UTC](https://discuss.elastic.co/t/could-not-load-fields-from-index-in-datafeed-while-creating-a-ml-job/220693/11 "2020-03-25T19:42:55Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.