I have an index with a nested field 'roles':
"roles": {
"type": "nested",
"properties": {
"name": {
"type": "text",
"fields": {
"raw": {
"type": "text",
"analyzer": "keylower"
}
}
},
"responsibilities": {
"properties": {
"name": {
"type": "text",
"fields": {
"raw": {
"type": "text",
"analyzer": "keylower"
}
}
}
}
}
}
}
The values in these fields are arrays, for eg.:
"roles": [
{
"name": "System Analyst",
"responsibilities": [
{
"name": "Software Development"
},
{
"name": "Software Testing"
}
]
},
{
"name": "Data Analyst",
"responsibilities": [
{
"name": "Data analysis"
},
{
"name": "Reporting"
}
]
}
]
I have to build Kibana visualizations on these fields separately. Since it is a nested field and kibana doesn't support it yet (?), I thought of creating runtime fields for each of these fields.
This is the query I used for roles:
PUT employee/_mappings
{
"runtime": {
"empRoles": {
"type": "keyword",
"script": """if (doc["roles.name.raw"].size()!=0 ) {
String[] empRoles;
for(int i=0; i < doc["roles.name.raw"].size(); i++) {
empRoles[i] = doc["roles.name.raw"].value ;
}
emit(empRoles);}"""
}
}
}
But I am getting error:
"caused_by" : {
"type" : "class_cast_exception",
"reason" : "Cannot cast from [java.lang.String[]] to [java.lang.String]."
}
How can I make it work? Ultimately, I want to build a kibana dashboard on the fields 'roles' and 'responsibilities'.
Hi @cr_168328
in order to produce an array of values in runtime fields you can emit multiple times a single value.
For more details about array values support for the runtime field emit there's this nice reference post: How to a emit a value for a runtime field of type geo_point? - #5 by rcowart
Hi @Marco_Liberati
As suggested, I modified the script as follows:
PUT employee/_mappings
{
"runtime": {
"empRoles": {
"type": "keyword",
"script": """if (doc["roles.name.raw"].size()!=0 ) {
for(int i=0; i < doc["roles.name.raw"].size(); i++) {
emit(doc["roles.name.raw"].value);
}
}"""
}
}
}
Now I am not getting any error, but neither am I getting the value in the field.
I performed the following search query:
GET /employee/_search?pretty
{
"_source": false,
"query": {
"match": {
"emailId": "abc@xyz.com"
}
},
"fields": [
"empRoles","roles.name.raw"
]
}
I got the following response:
{
"took" : 8,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 1,
"relation" : "eq"
},
"max_score" : 12.575342,
"hits" : [
{
"_index" : "employee",
"_type" : "_doc",
"_id" : "0bb26551-dfeb-4fbd-9c37-96016894b843",
"_score" : 12.575342,
"fields" : {
"roles" : [
{
"name.raw" : [
"System Analyst"
]
}
]
}
}
]
}
}
The runtime field was not populated.
Please guide on where I am doing it wrong.
The field types defined are of type text which cannot use the aggregation API to build visualizations.keyword where and then use a runtime field to build a visualization.
For more information on what type of fields Lens (and other viz editors) support, you can refer to this page (FAW at the bottom): Create visualizations with Lens | Kibana Guide [8.5] | Elastic
As last resort, you can use Vega where the results of your raw query need to be manipulated manually.
I have defined the field roles.name.raw as keyword (deleted the index, created new index and did reindexing)
{
"mappings": {
"dynamic": "false",
"runtime": {
"empRoles": {
"type": "keyword",
"script": {
"source": """if (doc["roles.name.raw"].size()!=0 ) {
for(int i=0; i < doc["roles.name.raw"].size(); i++) {
emit( doc["roles.name.raw"].value);
}
}""",
"lang": "painless"
}
}
},
"properties": {
"roles": {
"type": "nested",
"properties": {
"responsibilities": {
"properties": {
"name": {
"type": "text",
"fields": {
"raw": {
"type": "keyword"
}
},
"copy_to": [
"all_fields"
],
"fielddata": true
}
}
},
"name": {
"type": "text",
"fields": {
"raw": {
"type": "keyword"
}
},
"copy_to": [
"all_fields"
],
"fielddata": true
}
}
}
}
}
}
I created the runtime field using the previous script, still the value is not getting populated.
Can you post the new mapping?
I've tested with a nested field with keywords locally and it worked fine.
I have posted the new mapping for the relevant fields above.
{
"mappings": {
"dynamic": "false",
"runtime": {
"empRoles": {
"type": "keyword",
"script": {
"source": """if (doc["roles.name.raw"].size()!=0 ) {
for(int i=0; i < doc["roles.name.raw"].size(); i++) {
emit( doc["roles.name.raw"].value);
}
}""",
"lang": "painless"
}
}
},
"properties": {
"roles": {
"type": "nested",
"properties": {
"responsibilities": {
"properties": {
"name": {
"type": "text",
"fields": {
"raw": {
"type": "keyword"
}
},
"copy_to": [
"all_fields"
],
"fielddata": true
}
}
},
"name": {
"type": "text",
"fields": {
"raw": {
"type": "keyword"
}
},
"copy_to": [
"all_fields"
],
"fielddata": true
}
}
}
}
}
}
Sorry, didn't see it before.object type rather than nested.nested field types with runtime field.nested structure via _source, but that will have some performance impact: ElasticSearch aggregate on a scripted nested field - #2 by doogyatnesta
I have tried using param._source as suggested, but still not working.
This is not populating the runtime field:
PUT employee/_mappings
{
"runtime": {
"empRoles": {
"type": "keyword",
"script": """if (doc["roles.name.raw"].size()!=0 ) {
for(item in params._source.roles.name.raw) {
emit(item) ;
}
}
"""
}
}
}
The following one results in error cannot cast from [java.lang.String[]] to [void], as expected:
PUT employee/_mappings
{
"runtime": {
"empRoles": {
"type": "keyword",
"script": """String[] empRoles;
if (doc["roles.name.raw"].size()!=0 ) {
int i=0;
for(item in params._source.roles.name.raw) {
empRoles[i++]=item
}
}
return empRoles;
"""
}
}
}
I've managed to achieve the array result defining a script field on the dataView as follow:
def names = new String[params._source.roles.length];
def i= 0;
for( role in params._source.roles){
if(role.name != null){
names[i++] = role.name;
}
}
return names;
Add this script via the Kibana management UI: DataViews > [your dataView] > Click on Scripted fieldstab >Add scripted field`.
Thank you for the effort !
It worked.
(I am using Kibana 7.17. I guess DataViews is not available in this version. I added the script at Index Pattern > [index pattern name] > Scripted fields tab > Add scripted field )
In newer versions Index Pattern has been renamed to Data View.
Sure.
Also, I could get the visualization on the field roles, but the same script is not giving result for the field responsibilities, which is a field inside roles.
If you want to collect all responsibilities flatten into a scripted field, you can use this script:
def responsibilitiesLength = 0;
for( role in params._source.roles){
responsibilitiesLength += role.responsibilities.length;
}
def responsibilities = new String[responsibilitiesLength];
def i= 0;
for( role in params._source.roles){
for( responsability in role.responsibilities){
responsibilities[i++] = responsability.name
}
}
return responsibilities;
and define it as new scripted field.
system
February 8, 2023, 10:53am
17
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
