# Issue with keyword aggregation following update to 6.0

**URL:** https://discuss.elastic.co/t/issue-with-keyword-aggregation-following-update-to-6-0/110113
**Category:** Kibana
**Created:** [December 4, 2017, 10:10am UTC](https://discuss.elastic.co/t/issue-with-keyword-aggregation-following-update-to-6-0/110113 "2017-12-04T10:10:22Z")
**Posts on this page:** 11
**Page:** 1

<div class="post-metadata">

### Author: ![MrTraan](https://avatars.discourse-cdn.com/v4/letter/m/58956e/32.png) [@MrTraan](https://discuss.elastic.co/u/MrTraan)
#### Post date: [December 4, 2017, 10:10am UTC](https://discuss.elastic.co/t/issue-with-keyword-aggregation-following-update-to-6-0/110113/1 "2017-12-04T10:10:22Z")

</div>

Hello world,

Since I updated Kibana and ES to 6.0, my visualizations that relies on aggregations do not work anymore.  
I see a message saying "No results displayed because all values equal 0." instead. If I select a time range that goes before the migrations they work perfectly fine.

If I run a search with the following body:

```auto
{
    "aggs" : {
        "intents" : {
            "terms" : {
              "field" : "intent.keyword",
              "size": 10
            }
        }
    }
}

```

It runs without any problem but it only aggregates documents that were already there before the migration.

I have already tried to refresh my index pattern but it hasn't helped.

I'm quite new to ELK and I feel a bit lost, sorry if I'm missing something obvious!

---

<div class="post-metadata">

### Author: ![weltenwort](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/weltenwort/32/53885_2.png) [@weltenwort](https://discuss.elastic.co/u/weltenwort)
#### Post date: [December 5, 2017, 3:42pm UTC](https://discuss.elastic.co/t/issue-with-keyword-aggregation-following-update-to-6-0/110113/2 "2017-12-05T15:42:04Z")

</div>

Hi @MrTraan,

I would need to find out more about your setup to narrow down the possible causes:

- In which visualization is the message being displayed?
- And are the new documents indexed into a new index?
- Is that index matched by the index pattern configure in Kibana?
- Does that index have the same mapping as the old ones?

---

<div class="post-metadata">

### Author: ![MrTraan](https://avatars.discourse-cdn.com/v4/letter/m/58956e/32.png) [@MrTraan](https://discuss.elastic.co/u/MrTraan)
#### Post date: [December 5, 2017, 5:06pm UTC](https://discuss.elastic.co/t/issue-with-keyword-aggregation-following-update-to-6-0/110113/3 "2017-12-05T17:06:11Z")

</div>

Hi @weltenwort and thanks for your reply,

- It happens in every visualizations that rely on aggregations by keyword terms. But it works still fine when they use the string format (for example I have graphs with filters like `intent:"something-*"`)
- The new documents are under the same index as the old one
- Yes
- Yes

One thing I am not sure about and don't know how to check, is if the new documents have their keyword field created properly.  
I don't really understand how keywords work yet, and I don't know if the keyword value of a string field is stored in elasticsearch or if it is just an abstract notion.

---

<div class="post-metadata">

### Author: ![weltenwort](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/weltenwort/32/53885_2.png) [@weltenwort](https://discuss.elastic.co/u/weltenwort)
#### Post date: [December 6, 2017, 10:01am UTC](https://discuss.elastic.co/t/issue-with-keyword-aggregation-following-update-to-6-0/110113/4 "2017-12-06T10:01:21Z")

</div>

If the documents are indexed into the same index, the type of the `intent.keyword` field is bound to be the same. Would it be possible for you to show me the mapping of that index (by running `GET indexname/_mapping` in the Kibana devtools with "indexname" replaced by the proper name)?

---

<div class="post-metadata">

### Author: ![MrTraan](https://avatars.discourse-cdn.com/v4/letter/m/58956e/32.png) [@MrTraan](https://discuss.elastic.co/u/MrTraan)
#### Post date: [December 6, 2017, 10:42am UTC](https://discuss.elastic.co/t/issue-with-keyword-aggregation-following-update-to-6-0/110113/5 "2017-12-06T10:42:08Z")

</div>

Sure:

```auto
{
  "xxx": {
    "mappings": {
      "logs": {
        "_all": {
          "enabled": true
        },
        "properties": {
          "@timestamp": {
            "type": "date"
          },
          "@version": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "index": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "intent": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "message_content": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "message_type": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "processing_time": {
            "type": "long"
          }
        }
      },
      "doc": {
        "_all": {
          "enabled": true
        },
        "properties": {
          "@timestamp": {
            "type": "date"
          },
          "@version": {
            "type": "text"
          },
          "index": {
            "type": "text"
          },
          "intent": {
            "type": "text"
          },
          "message_content": {
            "type": "text"
          },
          "message_type": {
            "type": "text"
          },
          "processing_time": {
            "type": "long"
          }
        }
      }
    }
  }
}

```

---

<div class="post-metadata">

### Author: ![weltenwort](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/weltenwort/32/53885_2.png) [@weltenwort](https://discuss.elastic.co/u/weltenwort)
#### Post date: [December 6, 2017, 10:58am UTC](https://discuss.elastic.co/t/issue-with-keyword-aggregation-following-update-to-6-0/110113/6 "2017-12-06T10:58:45Z")

</div>

Of what type are the new documents? It looks like the mapping for the type `doc` does not contain the `intent.keyword` subfield.

(As an aside please note that mapping types have been deprecated in 6.0.0. While they will continue to work for indices created before 6.0, but will not be supported from 7.0 onwards. Please see [https://www.elastic.co/guide/en/elasticsearch/reference/current/removal-of-types.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/removal-of-types.html) for migration strategies.)

---

<div class="post-metadata">

### Author: ![MrTraan](https://avatars.discourse-cdn.com/v4/letter/m/58956e/32.png) [@MrTraan](https://discuss.elastic.co/u/MrTraan)
#### Post date: [December 6, 2017, 11:10am UTC](https://discuss.elastic.co/t/issue-with-keyword-aggregation-following-update-to-6-0/110113/7 "2017-12-06T11:10:43Z")

</div>

How can I check the type of the new documents?

---

<div class="post-metadata">

### Author: ![MrTraan](https://avatars.discourse-cdn.com/v4/letter/m/58956e/32.png) [@MrTraan](https://discuss.elastic.co/u/MrTraan)
#### Post date: [December 6, 2017, 12:32pm UTC](https://discuss.elastic.co/t/issue-with-keyword-aggregation-following-update-to-6-0/110113/8 "2017-12-06T12:32:38Z")

</div>

Oh sorry @weltenwort I figured it out myself!  
Indeed the new documents have `_type:doc` while the old ones have type `_type:logs`.  
I now understand that the issue is that the mapping for type doc does not have keyword subfields.  
Am I correct if I say that sending the following request:

```auto
PUT my_index
{
	"mappings": {
	  "logs": {
	    "_all": {
	      "enabled": true
	    },
	    "properties": {
	      "@timestamp": {
	        "type": "date"
	      },
	      "@version": {
	        "type": "text",
	        "fields": {
	          "keyword": {
	            "type": "keyword",
	            "ignore_above": 256
	          }
	        }
	      },
	      "index": {
	        "type": "text",
	        "fields": {
	          "keyword": {
	            "type": "keyword",
	            "ignore_above": 256
	          }
	        }
	      },
	      "intent": {
	        "type": "text",
	        "fields": {
	          "keyword": {
	            "type": "keyword",
	            "ignore_above": 256
	          }
	        }
	      },
	      "message_content": {
	        "type": "text",
	        "fields": {
	          "keyword": {
	            "type": "keyword",
	            "ignore_above": 256
	          }
	        }
	      },
	      "message_type": {
	        "type": "text",
	        "fields": {
	          "keyword": {
	            "type": "keyword",
	            "ignore_above": 256
	          }
	        }
	      },
	      "processing_time": {
	        "type": "long"
	      }
	    }
	  },
	  "doc": {
	    "_all": {
	      "enabled": true
	    },
	    "properties": {
	      "@timestamp": {
          "type": "date"
        },
        "@version": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "index": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "intent": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "message_content": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "message_type": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "processing_time": {
          "type": "long"
        }
	    }
	  }
	}
}

```

Should fix it?

And do you know why the type of my documents have changed? I didn't do it on purpose

---

<div class="post-metadata">

### Author: ![MrTraan](https://avatars.discourse-cdn.com/v4/letter/m/58956e/32.png) [@MrTraan](https://discuss.elastic.co/u/MrTraan)
#### Post date: [December 6, 2017, 12:59pm UTC](https://discuss.elastic.co/t/issue-with-keyword-aggregation-following-update-to-6-0/110113/9 "2017-12-06T12:59:38Z")

</div>

Yay I figured it all out!

If anyone come across this post I had to the following request:

```auto
PUT /my_index/mappings/doc
{
  "_all": {
    "enabled": true
  },
  "properties": {
    "@timestamp": {
      "type": "date"
    },
    "@version": {
      "type": "text",
      "fields": {
        "keyword": {
          "type": "keyword",
          "ignore_above": 256
        }
      }
    },
    "index": {
      "type": "text",
      "fields": {
        "keyword": {
          "type": "keyword",
          "ignore_above": 256
        }
      }
    },
    "intent": {
      "type": "text",
      "fields": {
        "keyword": {
          "type": "keyword",
          "ignore_above": 256
        }
      }
    },
    "message_content": {
      "type": "text",
      "fields": {
        "keyword": {
          "type": "keyword",
          "ignore_above": 256
        }
      }
    },
    "message_type": {
      "type": "text",
      "fields": {
        "keyword": {
          "type": "keyword",
          "ignore_above": 256
        }
      }
    },
    "processing_time": {
      "type": "long"
    }
  }
}

```

And then I did the following request to apply the mapping to my stored documents:

```auto
POST /my_index/_update_by_query?pretty&conflicts=proceed&refresh

```

Thanks a lot @weltenwort for your help!

---

<div class="post-metadata">

### Author: ![weltenwort](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/weltenwort/32/53885_2.png) [@weltenwort](https://discuss.elastic.co/u/weltenwort)
#### Post date: [December 7, 2017, 5:48pm UTC](https://discuss.elastic.co/t/issue-with-keyword-aggregation-following-update-to-6-0/110113/10 "2017-12-07T17:48:26Z")

</div>

Thanks for sharing your solution. As to why the document type has changed, it is probably a change in your ingestion pipeline (beats, logstash, ...?). The removal of mapping types I linked to might be the underlying reason.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 4, 2018, 5:48pm UTC](https://discuss.elastic.co/t/issue-with-keyword-aggregation-following-update-to-6-0/110113/11 "2018-01-04T17:48:28Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
