# \_update\_by\_query can't fill subfields in some documents

**URL:** <https://discuss.elastic.co/t/update-by-query-cant-fill-subfields-in-some-documents/277804>\
**Category:** Elasticsearch\
**Created:** [July 5, 2021, 9:46am UTC](https://discuss.elastic.co/t/update-by-query-cant-fill-subfields-in-some-documents/277804 "2021-07-05T09:46:19Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![\_anton](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/_anton/32/77134_2.png) [@\_anton](https://discuss.elastic.co/u/_anton)\
**Post date:** [July 5, 2021, 9:46am UTC](https://discuss.elastic.co/t/update-by-query-cant-fill-subfields-in-some-documents/277804/1 "2021-07-05T09:46:19Z")

</div>

After dinamically updating index mapping (I added new subfield with a lowercase normaliizer), I ran an \_update\_by\_query task to fill subfield with normalized values. However the task doesn't create subfield for several documents.

I added subfield "useLowercase" for field "canonical":

```auto
{
  "my_index" : {
    "mappings" : {
      "properties" : {
        "_class" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "canonical" : {
          "type" : "keyword",
          "fields" : {
            "keywordLowercase" : {
              "type" : "keyword",
              "ignore_above" : 256,
              "normalizer" : "lower_case_normalizer"
            },
            "useLowercase" : {
              "type" : "keyword",
              "ignore_above" : 256,
              "normalizer" : "useLowercase"
            }
          }
        },
        "certificationList" : {
          "properties" : {
....

```

Normalizer:

```auto
      "analysis" : {
          "normalizer" : {
...
            "useLowercase" : {
              "filter" : [
                "lowercase"
              ],
              "type" : "custom"
            }
          },
...

```

After that i ran \_update\_by\_query task:

```auto
POST my_index/_update_by_query?slices=auto&conflicts=proceed&wait_for_completion=false
{
  "query": {
    "bool": {
      "must_not": [
        {
          "exists": {
            "field": "canonical.useLowercase"
          }
        }
      ]
    }
  }
}

```

The task updated almost all documents and was completed with no failures. However it left about 4.5k documents with empty canonical.useLowercase field. Query:

```auto
GET my_index/_search?track_total_hits=true
{
  "query": {
    "bool": {
      "must_not": [
        {
          "exists": {
            "field": "canonical.useLowercase"
          }
        }
      ],
      "must": [
        {
          "exists": {
            "field": "canonical"
          }
        }
      ]
    }
  }
} 

```

Starting \_update\_by\_query one more time gave no result.

Question: why does it happen and how to update the rest of the documents?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 2, 2021, 9:46am UTC](https://discuss.elastic.co/t/update-by-query-cant-fill-subfields-in-some-documents/277804/2 "2021-08-02T09:46:34Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.