# How to create an array runtime field

**URL:** <https://discuss.elastic.co/t/how-to-create-an-array-runtime-field/322791>\
**Category:** Kibana\
**Created:** [January 10, 2023, 6:27am UTC](https://discuss.elastic.co/t/how-to-create-an-array-runtime-field/322791 "2023-01-10T06:27:37Z")\
**Posts on this page:** 17\
**Page:** 1

<div class="post-metadata">

**Author:** ![cr\_168328](https://avatars.discourse-cdn.com/v4/letter/c/76d3ee/32.png) [@cr\_168328](https://discuss.elastic.co/u/cr_168328)\
**Post date:** [January 10, 2023, 6:27am UTC](https://discuss.elastic.co/t/how-to-create-an-array-runtime-field/322791/1 "2023-01-10T06:27:37Z")

</div>

I have an index with a nested field 'roles':

```auto
"roles": {
    "type": "nested",
    "properties": {
        "name": {
            "type": "text",
            "fields": {
                "raw": {
                    "type": "text",
                    "analyzer": "keylower"
                }
            }
        },
        "responsibilities": {
            "properties": {
                "name": {
                    "type": "text",
                    "fields": {
                        "raw": {
                            "type": "text",
                            "analyzer": "keylower"
                        }
                    }
                }
            }
        }
    }
}

```

The values in these fields are arrays, for eg.:

```auto
"roles": [
        {
            "name": "System Analyst",
            "responsibilities": [
                {
                    "name": "Software Development"
                },
                {
                    "name": "Software Testing"
                }
            ]
        },
        {
            "name": "Data Analyst",
            "responsibilities": [
                {
                    "name": "Data analysis"
                },
                {
                    "name": "Reporting"
                }
            ]
        }
    ]

```

I have to build Kibana visualizations on these fields separately. Since it is a nested field and kibana doesn't support it yet (?), I thought of creating runtime fields for each of these fields.

This is the query I used for roles:

```auto
PUT employee/_mappings
{
  "runtime": {
    "empRoles": {
      "type": "keyword",
      "script": """if (doc["roles.name.raw"].size()!=0 ) {
        String[] empRoles;
        for(int i=0; i < doc["roles.name.raw"].size(); i++) {
          empRoles[i] = doc["roles.name.raw"].value ;
          
        }
         emit(empRoles);}"""
    }
  }
}

```

But I am getting error:

```auto
"caused_by" : {
        "type" : "class_cast_exception",
        "reason" : "Cannot cast from [java.lang.String[]] to [java.lang.String]."
      }

```

How can I make it work? Ultimately, I want to build a kibana dashboard on the fields 'roles' and 'responsibilities'.

---

<div class="post-metadata">

**Author:** ![Marco\_Liberati](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marco_liberati/32/82953_2.png) [@Marco\_Liberati](https://discuss.elastic.co/u/Marco_Liberati)\
**Post date:** [January 10, 2023, 8:49am UTC](https://discuss.elastic.co/t/how-to-create-an-array-runtime-field/322791/2 "2023-01-10T08:49:02Z")

</div>

Hi @cr_168328

in order to produce an array of values in runtime fields you can emit multiple times a single value.  
In your case rather than populate an array of value in the loop, just emit the value.

For more details about array values support for the runtime field `emit` there's this nice reference post: [How to a emit a value for a runtime field of type geo\_point? - #5 by rcowart](https://discuss.elastic.co/t/how-to-a-emit-a-value-for-a-runtime-field-of-type-geo-point/271889/5)

---

<div class="post-metadata">

**Author:** ![cr\_168328](https://avatars.discourse-cdn.com/v4/letter/c/76d3ee/32.png) [@cr\_168328](https://discuss.elastic.co/u/cr_168328)\
**Post date:** [January 10, 2023, 9:19am UTC](https://discuss.elastic.co/t/how-to-create-an-array-runtime-field/322791/3 "2023-01-10T09:19:05Z")

</div>

Hi @Marco_Liberati

As suggested, I modified the script as follows:

```auto
PUT employee/_mappings
{
  "runtime": {
    "empRoles": {
      "type": "keyword",
      "script": """if (doc["roles.name.raw"].size()!=0 ) {
       for(int i=0; i < doc["roles.name.raw"].size(); i++) {
          emit(doc["roles.name.raw"].value);
        }
         }"""
    }
  }
}

```

Now I am not getting any error, but neither am I getting the value in the field.

I performed the following search query:

```auto
GET /employee/_search?pretty
{
  "_source": false, 
  "query": {
          "match": {
            "emailId": "abc@xyz.com"
          }    
  },
  "fields": [
    "empRoles","roles.name.raw"
  ]
}

```

I got the following response:

```auto
{
  "took" : 8,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 1,
      "relation" : "eq"
    },
    "max_score" : 12.575342,
    "hits" : [
      {
        "_index" : "employee",
        "_type" : "_doc",
        "_id" : "0bb26551-dfeb-4fbd-9c37-96016894b843",
        "_score" : 12.575342,
        "fields" : {
          "roles" : [
            {
              "name.raw" : [
                "System Analyst"
              ]
            }
          ]
        }
      }
    ]
  }
}

```

The runtime field was not populated.

Please guide on where I am doing it wrong.

---

<div class="post-metadata">

**Author:** ![Marco\_Liberati](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marco_liberati/32/82953_2.png) [@Marco\_Liberati](https://discuss.elastic.co/u/Marco_Liberati)\
**Post date:** [January 10, 2023, 10:16am UTC](https://discuss.elastic.co/t/how-to-create-an-array-runtime-field/322791/4 "2023-01-10T10:16:47Z")

</div>

The field types defined are of type `text` which cannot use the aggregation API to build visualizations.  
You need to define them as `keyword` where and then use a runtime field to build a visualization.

For more information on what type of fields Lens (and other viz editors) support, you can refer to this page (FAW at the bottom): [Create visualizations with Lens | Kibana Guide [8.5] | Elastic](https://www.elastic.co/guide/en/kibana/current/lens.html)

As last resort, you can use Vega where the results of your raw query need to be manipulated manually.

---

<div class="post-metadata">

**Author:** ![cr\_168328](https://avatars.discourse-cdn.com/v4/letter/c/76d3ee/32.png) [@cr\_168328](https://discuss.elastic.co/u/cr_168328)\
**Post date:** [January 10, 2023, 11:40am UTC](https://discuss.elastic.co/t/how-to-create-an-array-runtime-field/322791/5 "2023-01-10T11:40:34Z")

</div>

I have defined the field `roles.name.raw` as keyword (deleted the index, created new index and did reindexing)

```auto
{
    "mappings": {
        "dynamic": "false",
        "runtime": {
            "empRoles": {
                "type": "keyword",
                "script": {
                    "source": """if (doc["roles.name.raw"].size()!=0 ) {
                    for(int i=0; i < doc["roles.name.raw"].size(); i++) {
                        emit( doc["roles.name.raw"].value);
                    }
                }""",
            "lang": "painless"
            }
        }
    },
    "properties": {
        "roles": {
            "type": "nested",
            "properties": {
                "responsibilities": {
                    "properties": {
                        "name": {
                            "type": "text",
                            "fields": {
                                "raw": {
                                    "type": "keyword"
                                }
                            },
                            "copy_to": [
                                "all_fields"
                            ],
                            "fielddata": true
                        }
                    }
                },
                "name": {
                    "type": "text",
                    "fields": {
                        "raw": {
                            "type": "keyword"
                        }
                    },
                    "copy_to": [
                        "all_fields"
                    ],
                    "fielddata": true
                }
            }
        }
    }
    }
}

```

I created the runtime field using the previous script, still the value is not getting populated.

---

<div class="post-metadata">

**Author:** ![Marco\_Liberati](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marco_liberati/32/82953_2.png) [@Marco\_Liberati](https://discuss.elastic.co/u/Marco_Liberati)\
**Post date:** [January 10, 2023, 11:41am UTC](https://discuss.elastic.co/t/how-to-create-an-array-runtime-field/322791/6 "2023-01-10T11:41:59Z")

</div>

Can you post the new mapping?

I've tested with a nested field with keywords locally and it worked fine. 🤔

---

<div class="post-metadata">

**Author:** ![cr\_168328](https://avatars.discourse-cdn.com/v4/letter/c/76d3ee/32.png) [@cr\_168328](https://discuss.elastic.co/u/cr_168328)\
**Post date:** [January 10, 2023, 11:42am UTC](https://discuss.elastic.co/t/how-to-create-an-array-runtime-field/322791/7 "2023-01-10T11:42:39Z")

</div>

I have posted the new mapping for the relevant fields above.

---

<div class="post-metadata">

**Author:** ![cr\_168328](https://avatars.discourse-cdn.com/v4/letter/c/76d3ee/32.png) [@cr\_168328](https://discuss.elastic.co/u/cr_168328)\
**Post date:** [January 10, 2023, 11:43am UTC](https://discuss.elastic.co/t/how-to-create-an-array-runtime-field/322791/8 "2023-01-10T11:43:55Z")

</div>

```auto
{
    "mappings": {
        "dynamic": "false",
        "runtime": {
            "empRoles": {
                "type": "keyword",
                "script": {
                    "source": """if (doc["roles.name.raw"].size()!=0 ) {
                    for(int i=0; i < doc["roles.name.raw"].size(); i++) {
                        emit( doc["roles.name.raw"].value);
                    }
                }""",
            "lang": "painless"
            }
        }
    },
    "properties": {
        "roles": {
            "type": "nested",
            "properties": {
                "responsibilities": {
                    "properties": {
                        "name": {
                            "type": "text",
                            "fields": {
                                "raw": {
                                    "type": "keyword"
                                }
                            },
                            "copy_to": [
                                "all_fields"
                            ],
                            "fielddata": true
                        }
                    }
                },
                "name": {
                    "type": "text",
                    "fields": {
                        "raw": {
                            "type": "keyword"
                        }
                    },
                    "copy_to": [
                        "all_fields"
                    ],
                    "fielddata": true
                }
            }
        }
    }
    }
}

```

---

<div class="post-metadata">

**Author:** ![Marco\_Liberati](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marco_liberati/32/82953_2.png) [@Marco\_Liberati](https://discuss.elastic.co/u/Marco_Liberati)\
**Post date:** [January 10, 2023, 1:28pm UTC](https://discuss.elastic.co/t/how-to-create-an-array-runtime-field/322791/9 "2023-01-10T13:28:38Z")

</div>

Sorry, didn't see it before.  
So I had it wrong when testing and used a regular `object` type rather than `nested`.  
I do think there's currently no way to use `nested` field types with runtime field.  
Based on this answer it seems that deprecated scripted field can provide some help manually accessing the `nested` structure via `_source`, but that will have some performance impact: [ElasticSearch aggregate on a scripted nested field - #2 by doogyatnesta](https://discuss.elastic.co/t/elasticsearch-aggregate-on-a-scripted-nested-field/298841/2)

---

<div class="post-metadata">

**Author:** ![cr\_168328](https://avatars.discourse-cdn.com/v4/letter/c/76d3ee/32.png) [@cr\_168328](https://discuss.elastic.co/u/cr_168328)\
**Post date:** [January 11, 2023, 6:23am UTC](https://discuss.elastic.co/t/how-to-create-an-array-runtime-field/322791/10 "2023-01-11T06:23:11Z")

</div>

> [@Marco\_Liberati](#):
>
> Based on this answer it seems that deprecated scripted field can provide some help

I have tried using `param._source` as suggested, but still not working.

This is not populating the runtime field:

```auto
PUT employee/_mappings
{
  "runtime": {
    "empRoles": {
      "type": "keyword",
      "script": """if (doc["roles.name.raw"].size()!=0 ) {
        for(item in params._source.roles.name.raw) {
          emit(item) ;
        }
        
         }
         """
    }
  }
}

```

The following one results in error `cannot cast from [java.lang.String[]] to [void]`, as expected:

```auto
PUT employee/_mappings
{
  "runtime": {
    "empRoles": {
      "type": "keyword",
      "script": """String[] empRoles;
        if (doc["roles.name.raw"].size()!=0 ) {
        int i=0;
        for(item in params._source.roles.name.raw) {
          empRoles[i++]=item
        }
        
         }
return empRoles;
         """
    }
  }
}

```

---

<div class="post-metadata">

**Author:** ![Marco\_Liberati](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marco_liberati/32/82953_2.png) [@Marco\_Liberati](https://discuss.elastic.co/u/Marco_Liberati)\
**Post date:** [January 11, 2023, 8:59am UTC](https://discuss.elastic.co/t/how-to-create-an-array-runtime-field/322791/11 "2023-01-11T08:59:46Z")

</div>

I've managed to achieve the array result defining a script field on the dataView as follow:

```auto
def names = new String[params._source.roles.length];
def i= 0;
for( role in params._source.roles){
  if(role.name != null){
    names[i++] = role.name;
  }
}
return names;

```

Add this script via the Kibana management UI: `DataViews > [your dataView] > Click on `Scripted fields`tab >`Add scripted field`.

---

<div class="post-metadata">

**Author:** ![cr\_168328](https://avatars.discourse-cdn.com/v4/letter/c/76d3ee/32.png) [@cr\_168328](https://discuss.elastic.co/u/cr_168328)\
**Post date:** [January 11, 2023, 10:12am UTC](https://discuss.elastic.co/t/how-to-create-an-array-runtime-field/322791/12 "2023-01-11T10:12:05Z")

</div>

Thank you for the effort !

It worked.

(I am using Kibana 7.17. I guess `DataViews` is not available in this version. I added the script at `Index Pattern > [index pattern name] > Scripted fields tab > Add scripted field` )

---

<div class="post-metadata">

**Author:** ![Marco\_Liberati](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marco_liberati/32/82953_2.png) [@Marco\_Liberati](https://discuss.elastic.co/u/Marco_Liberati)\
**Post date:** [January 11, 2023, 10:16am UTC](https://discuss.elastic.co/t/how-to-create-an-array-runtime-field/322791/13 "2023-01-11T10:16:53Z")

</div>

In newer versions `Index Pattern` has been renamed to `Data View`.  
Just mind that scripted fields can incur in performance issues and have been deprecated for this reason.

---

<div class="post-metadata">

**Author:** ![cr\_168328](https://avatars.discourse-cdn.com/v4/letter/c/76d3ee/32.png) [@cr\_168328](https://discuss.elastic.co/u/cr_168328)\
**Post date:** [January 11, 2023, 10:31am UTC](https://discuss.elastic.co/t/how-to-create-an-array-runtime-field/322791/14 "2023-01-11T10:31:37Z")

</div>

Sure.

Also, I could get the visualization on the field `roles`, but the same script is not giving result for the field `responsibilities`, which is a field inside `roles`.

---

<div class="post-metadata">

**Author:** ![Marco\_Liberati](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marco_liberati/32/82953_2.png) [@Marco\_Liberati](https://discuss.elastic.co/u/Marco_Liberati)\
**Post date:** [January 11, 2023, 10:40am UTC](https://discuss.elastic.co/t/how-to-create-an-array-runtime-field/322791/15 "2023-01-11T10:40:32Z")

</div>

If you want to collect all `responsibilities` flatten into a scripted field, you can use this script:

```auto
def responsibilitiesLength = 0;
for( role in params._source.roles){
    responsibilitiesLength += role.responsibilities.length;
}
def responsibilities = new String[responsibilitiesLength];
def i= 0;
for( role in params._source.roles){
    for( responsability in role.responsibilities){
      responsibilities[i++] = responsability.name
    }
}
return responsibilities;

```

and define it as new scripted field.

---

<div class="post-metadata">

**Author:** ![cr\_168328](https://avatars.discourse-cdn.com/v4/letter/c/76d3ee/32.png) [@cr\_168328](https://discuss.elastic.co/u/cr_168328)\
**Post date:** [January 11, 2023, 10:53am UTC](https://discuss.elastic.co/t/how-to-create-an-array-runtime-field/322791/16 "2023-01-11T10:53:23Z")

</div>

Thank you!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 8, 2023, 10:53am UTC](https://discuss.elastic.co/t/how-to-create-an-array-runtime-field/322791/17 "2023-02-08T10:53:43Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.