# Mutate filter plugin: convert to keyword and text?

**URL:** <https://discuss.elastic.co/t/mutate-filter-plugin-convert-to-keyword-and-text/66936>\
**Category:** Logstash\
**Created:** [November 23, 2016, 3:37am UTC](https://discuss.elastic.co/t/mutate-filter-plugin-convert-to-keyword-and-text/66936 "2016-11-23T03:37:52Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![GrahamHannington](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/grahamhannington/32/4404_2.png) [@GrahamHannington](https://discuss.elastic.co/u/GrahamHannington)\
**Post date:** [November 23, 2016, 3:37am UTC](https://discuss.elastic.co/t/mutate-filter-plugin-convert-to-keyword-and-text/66936/1 "2016-11-23T03:37:52Z")

</div>

From the current [mutate convert documentation](https://www.elastic.co/guide/en/logstash/current/plugins-filters-mutate.html#plugins-filters-mutate-convert):

> Valid conversion targets are: integer, float, string, and boolean.

I expected to see the new-for-5.0 keyword and text types in that list. And perhaps a note about the string type being [deprecated](https://www.elastic.co/guide/en/elasticsearch/reference/5.0/breaking_50_mapping_changes.html#_literal_string_literal_fields_replaced_by_literal_text_literal_literal_keyword_literal_fields).

I was about to create an issue for the [logstash-filter-mutate](https://github.com/logstash-plugins/logstash-filter-mutate/issues) GitHub repo about this, but after reading the following request in the placeholder text for the new issue:

> Please post all product and debugging questions on our forum

I decided to create this forum topic.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [November 23, 2016, 6:46am UTC](https://discuss.elastic.co/t/mutate-filter-plugin-convert-to-keyword-and-text/66936/2 "2016-11-23T06:46:29Z")

</div>

You're conflating the types of fields in Logstash events with the mappings in Elasticsearch. They are entirely different things. You should think of Logstash events as JSON documents, and JSON values are either objects, arrays, strings, numbers, or booleans. How these values are mapped in Elasticsearch is decided by Elasticsearch and is nothing Logstash can affect except via the index template (and indirectly by converting fields to a JSON data type that allows ES's dynamic mapper to do the right thing).

---

<div class="post-metadata">

**Author:** ![GrahamHannington](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/grahamhannington/32/4404_2.png) [@GrahamHannington](https://discuss.elastic.co/u/GrahamHannington)\
**Post date:** [November 23, 2016, 8:36am UTC](https://discuss.elastic.co/t/mutate-filter-plugin-convert-to-keyword-and-text/66936/3 "2016-11-23T08:36:48Z")

</div>

D'oh! Thank you. Yes, I see what you mean. And, yes, you're right about what I was thinking. I'm embarrassed, but grateful for the correction.

Using `mutate` to convert the data type of a field to the Elasticsearch-specific `keyword` or `text` type was an idea (a wrongheaded one, as you've pointed out) that occurred to me as a possible alternative to what I'm actually doing, which is to use an index template.

## What I was doing in Elastic \< 5.0

In Elastic 1._x_ and 2._x_, I used the following index template to set all string fields in the index pattern fuw-\* to `not_analyzed`:

```
{
  "template" : "fuw-*", 1 
  "mappings" : {
    "_default_" : {
    "dynamic_templates" : [ {
      "string_fields" : {
      "mapping" : {
        "index" : "not_analyzed",
        "omit_norms" : true,
        "type" : "string"
      },
      "match_mapping_type" : "string",
      "match" : "*"
      }
    } ]
    }
  }
}

```

(This works for me, and I'll likely still need to use/support Elasticsearch 2._x_, so I'd gratefully accept tips.)

## What I'm doing now in Elastic 5.0

In Elastic 5.0, I understand that the `index` property no longer supports the value `not_analyzed`.

So I'm now using the following index template to map "incoming" strings to the new-for-5.0 `keyword` type:

```
{
  "template": "fuw-*",
  "mappings": {
    "_default_": {
      "dynamic_templates": [{
        "string_fields": {
          "match": "*",
          "match_mapping_type": "string",
          "mapping": {
            "type": "keyword"
          }
        }
      }]
    }
  }
}

```

This seems to be working: for example, after loading data into that index pattern, Kibana doesn't show separate "keyword" fields in that index pattern, and I can successfully search on the "original" (non-analyzed) values.

But I'm left with a lingering doubt: in Management / Indices / Index Patterns, Kibana still shows the types of these fields in that index pattern as `string`.

I expected to see these fields as type `keyword`.

From the description at the top of that Kibana page:

> This page lists every field in the fuw-\* index and the field's associated core type as recorded by Elasticsearch.

I Googled for "core type", and found the Elastic documentation topic "[Field datatypes](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-types.html)":

> ### Core datatypes
> 
> **string**  
> text and keyword

Based on that documentation, perhaps everything's working as designed, and Kibana is correctly reporting `string` as the core type, where `text` and `keyword` are types that "belong" to that core type?

Or am I, once again, conflating types from different contexts?

---

<div class="post-metadata">

**Author:** ![GrahamHannington](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/grahamhannington/32/4404_2.png) [@GrahamHannington](https://discuss.elastic.co/u/GrahamHannington)\
**Post date:** [November 23, 2016, 9:36am UTC](https://discuss.elastic.co/t/mutate-filter-plugin-convert-to-keyword-and-text/66936/4 "2016-11-23T09:36:49Z")

</div>

My apology for my poor forum etiquette: it belatedly occurred to me that I should have closed this topic, and created a new topic for my "follow-on" question about Kibana reporting the "core type" as `string`. I'll do that - create a new topic - if asked, no problem.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [November 24, 2016, 8:39am UTC](https://discuss.elastic.co/t/mutate-filter-plugin-convert-to-keyword-and-text/66936/5 "2016-11-24T08:39:38Z")

</div>

I'm not 100% sure if keyword fields are supposed to be reported as strings. Asking in the Kibana or Elasticsearch group is probably better.

---

<div class="post-metadata">

**Author:** ![GrahamHannington](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/grahamhannington/32/4404_2.png) [@GrahamHannington](https://discuss.elastic.co/u/GrahamHannington)\
**Post date:** [November 25, 2016, 8:21am UTC](https://discuss.elastic.co/t/mutate-filter-plugin-convert-to-keyword-and-text/66936/6 "2016-11-25T08:21:36Z")

</div>

> [@magnusbaeck](#):
>
> Asking in the Kibana or Elasticsearch group is probably better.

Done (thanks for the tip):

[Kibana 5.0 shows field type as string, not keyword: working as designed?](https://discuss.elastic.co/t/kibana-5-0-shows-field-type-as-string-not-keyword-working-as-designed/67164)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 23, 2016, 8:22am UTC](https://discuss.elastic.co/t/mutate-filter-plugin-convert-to-keyword-and-text/66936/7 "2016-12-23T08:22:30Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.