# Unable to make a field aggregatable in kibana

**URL:** <https://discuss.elastic.co/t/unable-to-make-a-field-aggregatable-in-kibana/161912>\
**Category:** Elasticsearch\
**Created:** [December 22, 2018, 8:16am UTC](https://discuss.elastic.co/t/unable-to-make-a-field-aggregatable-in-kibana/161912 "2018-12-22T08:16:41Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![sid\_nikhil](https://avatars.discourse-cdn.com/v4/letter/s/f9ae1b/32.png) [@sid\_nikhil](https://discuss.elastic.co/u/sid_nikhil)\
**Post date:** [December 22, 2018, 8:16am UTC](https://discuss.elastic.co/t/unable-to-make-a-field-aggregatable-in-kibana/161912/1 "2018-12-22T08:16:42Z")

</div>

I have a field called "message" . I need to make it as aggregatable. I am unable to do so .  
Can someone please suggest how to achieve that.

![Capture_kibana_aggregation](https://us1.discourse-cdn.com/elastic/original/3X/e/4/e4e4bb234b6bd38325536a9795f7563d99b2e825.png)

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [December 22, 2018, 8:46am UTC](https://discuss.elastic.co/t/unable-to-make-a-field-aggregatable-in-kibana/161912/2 "2018-12-22T08:46:44Z")

</div>

Use a keyword type in your mapping.  
Or use `doc_values` for this field.

> [@sid\_nikhil](#):
>
> I request for immediate help from elastic team

You can't ask for that on a public forum manned by volunteers.

Read [this](https://discuss.elastic.co/t/about-the-elasticsearch-category/21) and specifically the "Also be patient" part.

It's fine to answer on your own thread after 2 or 3 days (not including weekends) if you don't have an answer.

---

<div class="post-metadata">

**Author:** ![sid\_nikhil](https://avatars.discourse-cdn.com/v4/letter/s/f9ae1b/32.png) [@sid\_nikhil](https://discuss.elastic.co/u/sid_nikhil)\
**Post date:** [December 22, 2018, 2:16pm UTC](https://discuss.elastic.co/t/unable-to-make-a-field-aggregatable-in-kibana/161912/3 "2018-12-22T14:16:10Z")

</div>

I did this in Dev tools  
PUT /filebeat  
{

```
      "mappings": {

        "_doc": {

          "properties": {

            "message": {

              "type": "keyword"

            }

          }

        }

      }

    }

```

But still message field is not aggregatable  
 ![Capture_kibana_aggregation](https://us1.discourse-cdn.com/elastic/original/3X/e/4/e4e4bb234b6bd38325536a9795f7563d99b2e825.png)

If my field is message , do you mean i should use message.doc\_values

This is my painless script to extract last word from message:  
String parts = /-/.split(doc['message'].value);  
return parts[-1]

Can you please explain with example . I am very new to ES/kibana.  
Thanks

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [December 22, 2018, 3:23pm UTC](https://discuss.elastic.co/t/unable-to-make-a-field-aggregatable-in-kibana/161912/4 "2018-12-22T15:23:06Z")

</div>

What you did is correct.  
If you reload the index in Kibana it will show that message is aggregatable.

---

<div class="post-metadata">

**Author:** ![sid\_nikhil](https://avatars.discourse-cdn.com/v4/letter/s/f9ae1b/32.png) [@sid\_nikhil](https://discuss.elastic.co/u/sid_nikhil)\
**Post date:** [December 22, 2018, 3:33pm UTC](https://discuss.elastic.co/t/unable-to-make-a-field-aggregatable-in-kibana/161912/5 "2018-12-22T15:33:05Z")

</div>

I reloaded the kibana dashboard & reloaded the index pattern as well , but still message field is not aggregatable as shown in above screenshot.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [December 22, 2018, 4:36pm UTC](https://discuss.elastic.co/t/unable-to-make-a-field-aggregatable-in-kibana/161912/6 "2018-12-22T16:36:30Z")

</div>

You need to click on

![image](https://us1.discourse-cdn.com/elastic/original/3X/2/7/2756ba8f59726a9f16f652a5c285bec50221f32c.jpeg)

If it does not work, start again from scratch (delete your existing index in elasticsearch and index template in Kibana).  
If it still does not work ask in #kibana and explain exactly all the steps you followed.

---

<div class="post-metadata">

**Author:** ![sid\_nikhil](https://avatars.discourse-cdn.com/v4/letter/s/f9ae1b/32.png) [@sid\_nikhil](https://discuss.elastic.co/u/sid_nikhil)\
**Post date:** [December 22, 2018, 4:59pm UTC](https://discuss.elastic.co/t/unable-to-make-a-field-aggregatable-in-kibana/161912/7 "2018-12-22T16:59:16Z")

</div>

I already cicked ithat refesh button .. didnt work

Then i deleted the index & created again .. still message field is not aggregatable

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [December 23, 2018, 5:42am UTC](https://discuss.elastic.co/t/unable-to-make-a-field-aggregatable-in-kibana/161912/8 "2018-12-23T05:42:44Z")

</div>

Start from scratch.  
Paste here every command you are running.

Also check with the GET mapping API what is the mapping for your index and check again the field

---

<div class="post-metadata">

**Author:** ![sid\_nikhil](https://avatars.discourse-cdn.com/v4/letter/s/f9ae1b/32.png) [@sid\_nikhil](https://discuss.elastic.co/u/sid_nikhil)\
**Post date:** [December 23, 2018, 6:02am UTC](https://discuss.elastic.co/t/unable-to-make-a-field-aggregatable-in-kibana/161912/9 "2018-12-23T06:02:02Z")

</div>

I tried this command in dev tools :  
GET /filebeat/\_mapping/\_doc

I got this result :  
{  
"filebeat" : {  
"mappings" : {  
"\_doc" : {  
"properties" : {  
"message" : {  
"type" : "keyword"  
}  
}  
}  
}  
}  
}  
This means message field has keyword type now . Still message field is non-aggregatable

 ![Capture_message](https://us1.discourse-cdn.com/elastic/original/3X/0/c/0c14258d0a244c12e6a2099e00335d5155a02a7b.png)

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [December 23, 2018, 10:37am UTC](https://discuss.elastic.co/t/unable-to-make-a-field-aggregatable-in-kibana/161912/10 "2018-12-23T10:37:44Z")

</div>

What steps are you doing in Kibana?  
Can you remove the index pattern in Kibana and add it again?

---

<div class="post-metadata">

**Author:** ![sid\_nikhil](https://avatars.discourse-cdn.com/v4/letter/s/f9ae1b/32.png) [@sid\_nikhil](https://discuss.elastic.co/u/sid_nikhil)\
**Post date:** [December 23, 2018, 12:50pm UTC](https://discuss.elastic.co/t/unable-to-make-a-field-aggregatable-in-kibana/161912/11 "2018-12-23T12:50:58Z")

</div>

These are the steps i followed . Pls correct me if anything is wrong  
1)removed index filebeat-\*  
2) I restarted filebeat service from srevices.msc and again created the index filebeat-\*  
3) I did this in the dev tools

```
PUT /filebeat
{

  "mappings": {

    "_doc": {

      "properties": {

        "message": {

          "type": "keyword"

        }

      }

    }

  }

}

```

Got the resource already exists exception:  
{  
"error": {  
"root\_cause": [  
{  
"type": "resource\_already\_exists\_exception",  
"reason": "index [filebeat/EDCdViXES3uOGaPJFUMv9A] already exists",  
"index\_uuid": "EDCdViXES3uOGaPJFUMv9A",  
"index": "filebeat"  
}  
],  
"type": "resource\_already\_exists\_exception",  
"reason": "index [filebeat/EDCdViXES3uOGaPJFUMv9A] already exists",  
"index\_uuid": "EDCdViXES3uOGaPJFUMv9A",  
"index": "filebeat"  
},  
"status": 400  
}

4)still i found that message field is not aggregatable .  
Is there any step which i might be missing ?

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [December 23, 2018, 3:13pm UTC](https://discuss.elastic.co/t/unable-to-make-a-field-aggregatable-in-kibana/161912/12 "2018-12-23T15:13:20Z")

</div>

It seems that you are using filebeat.

When you run:

```auto
PUT /filebeat
...

```

You are creating an index named filebeat. Which is not the name filebeat is using by default which is `filebeat-(timestamp)`.

That's probably why you can't see that in Kibana index settings where you have `filebeat-*` as the index names. This does not match `filebeat`.

I can definitely help to fix that but now as I understand that you are totally new to Elastic stack, I wonder if you really want to do aggregation on the filebeat `message` field.

What do you have in this field?  
What do you want to aggregate and what do you expect?

---

<div class="post-metadata">

**Author:** ![sid\_nikhil](https://avatars.discourse-cdn.com/v4/letter/s/f9ae1b/32.png) [@sid\_nikhil](https://discuss.elastic.co/u/sid_nikhil)\
**Post date:** [December 23, 2018, 4:25pm UTC](https://discuss.elastic.co/t/unable-to-make-a-field-aggregatable-in-kibana/161912/13 "2018-12-23T16:25:34Z")

</div>

Thanks David for response

My answer to why I am trying to aggregate message field ?

Ans:  
I have 2 fields of **string** type in filebeat index:

1. **\_index** = kibana\_sample\_data\_flights  
**(This field is marked aggregatable by default)**

2. **message** = i\_want\_to\_extract\_first\_word\_from\_this\_string  
**(This field is marked non-aggregatable by default)**

Now I create a **scripted field** called **firstword** (i am splitting the string using underscore as delimiter and getting the firstword)

When i try for **\_index** field, **the _painless query_ works as expected:**

```
String[] parts = /_/.split(doc['_index'].value);
return parts[0]

```

But When i try for **message** field, **i get error ( 3 of 6 shards failed) when in click on Discover:**

```
String[] parts = /_/.split(doc['message'].value);
return parts[0]

```

**The only difference i see between \_index and message is \_index is aggregatable but message is not .** So I wanted to make message aggregatable.

**Updates : I have been able to make the message field aggregatable , but it did not solve my above mentioned purpose of splitting and getting first word**

Now When i try for **message** field, **i get error ( 3 of 11 shards failed) when in click on Discover**

```
String[] parts = /_/.split(doc['message'].value);
return parts[0]

```

As you already know i am a beginner in ELK . Can you please help me resolve this.

Thanks a lot in advance

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [December 25, 2018, 6:18pm UTC](https://discuss.elastic.co/t/unable-to-make-a-field-aggregatable-in-kibana/161912/15 "2018-12-25T18:18:24Z")

</div>

I don't think that's the right way to do what you want to achieve.

You should better use an ingest pipeline which will extract the data you need at index time to a dedicated field where you can set it as a `keyword`.

---

<div class="post-metadata">

**Author:** ![sid\_nikhil](https://avatars.discourse-cdn.com/v4/letter/s/f9ae1b/32.png) [@sid\_nikhil](https://discuss.elastic.co/u/sid_nikhil)\
**Post date:** [December 26, 2018, 2:16am UTC](https://discuss.elastic.co/t/unable-to-make-a-field-aggregatable-in-kibana/161912/16 "2018-12-26T02:16:06Z")

</div>

1)Can u please guide me how to do that with a code snippet .  
Because all i want to do is split a sting , i guess it should not be this complicated.

2)Also :By Extracting to a dedicated field , do you mean scripted field ?  
(I am not using logstash)

3)Also i think there is something special with the **message** field beacause :  
Even this query gives error **( 3 of 14 shards failed) when in click on Discover:**

`return doc['message'].value;`

my actual message field is

**message** : 2018-12-21 02:31:31,792;INFO ;XSYD.2.5.0.1a5e8-uye1-9d87-8744-5343db306cd8;1;0;;GETCONFPRO;0;

I want to split by ; and get the timestamp

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [December 26, 2018, 6:05am UTC](https://discuss.elastic.co/t/unable-to-make-a-field-aggregatable-in-kibana/161912/17 "2018-12-26T06:05:38Z")

</div>

Then you probably want to use [https://www.elastic.co/guide/en/elasticsearch/reference/6.5/dissect-processor.html](https://www.elastic.co/guide/en/elasticsearch/reference/6.5/dissect-processor.html) or

> **[johtani/elasticsearch-ingest-csv](https://github.com/johtani/elasticsearch-ingest-csv)**
>
> Ingest CSV processor parses CSV data and stores it as individual fields - johtani/elasticsearch-ingest-csv

---

<div class="post-metadata">

**Author:** ![sid\_nikhil](https://avatars.discourse-cdn.com/v4/letter/s/f9ae1b/32.png) [@sid\_nikhil](https://discuss.elastic.co/u/sid_nikhil)\
**Post date:** [December 26, 2018, 6:18am UTC](https://discuss.elastic.co/t/unable-to-make-a-field-aggregatable-in-kibana/161912/18 "2018-12-26T06:18:19Z")

</div>

In the previous reply you suggested to use ingest pipeline ,  
can you explain how to do that ?

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [December 26, 2018, 1:18pm UTC](https://discuss.elastic.co/t/unable-to-make-a-field-aggregatable-in-kibana/161912/19 "2018-12-26T13:18:10Z")

</div>

If you just the README of the latest link I shared, you have some examples.

Otherwise start by reading the documentation :

[https://www.elastic.co/guide/en/elasticsearch/reference/6.5/ingest.html](https://www.elastic.co/guide/en/elasticsearch/reference/6.5/ingest.html)

---

<div class="post-metadata">

**Author:** ![sid\_nikhil](https://avatars.discourse-cdn.com/v4/letter/s/f9ae1b/32.png) [@sid\_nikhil](https://discuss.elastic.co/u/sid_nikhil)\
**Post date:** [December 26, 2018, 2:13pm UTC](https://discuss.elastic.co/t/unable-to-make-a-field-aggregatable-in-kibana/161912/20 "2018-12-26T14:13:40Z")

</div>

For that GITHUB link you shared , the setup given is for the mac os and those command does'nt run on windows .

When i run

`.\gradlew clean check`

I get error  
.\gradlew is not recognised as iternal or external command

I can't find installation procedure for windows there . Can you please check once

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [December 26, 2018, 9:38pm UTC](https://discuss.elastic.co/t/unable-to-make-a-field-aggregatable-in-kibana/161912/21 "2018-12-26T21:38:07Z")

</div>

Read this part of the documentation.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/9/9/9924e0d0d95114930fc43bd03df16ab4433a7cbc.jpeg)

[Next page](https://discuss.elastic.co/t/unable-to-make-a-field-aggregatable-in-kibana/161912.md?page=2)