I would like some assistance with creating custom fields in Kibana.
I am not sure why the filed I want is shown correctly in Discovery but when I want to Visualize it it changes back to how it is by default.
I am not sure how I need to script or format it so it gives me the value I need.
Also is it possible for me to use the URL and combine it with few other values.
I want the URL + id + parent_ruiid
All this combined and as a new filed, I tried Scripting fields but it does not work.
Hey @marija.timovska. Which version of kibana do you use and where do you try to visualize it and you can't see this formatted? I mean in which visualization type. Is this a table?
Oh I see. Enhanced datatable is an external plugin so I assume that is something that you have to report there. GitHub - fbaligand/kibana-enhanced-table: Kibana visualization like a Data Table, but with enhanced features like computed columns, filter bar, and “Split Cols” bucket
I have created the following scripted field
and it visualizes correctly on our datatable
{
"took": 13,
"timed_out": false,
"_shards": {
"total": 4,
"successful": 2,
"skipped": 2,
"failed": 2,
"failures": [
{
"shard": 0,
"index": "snaplogic_2021",
"node": "TjZg_rNkQd-OL5wYTsAg1Q",
"reason": {
"type": "script_exception",
"reason": "runtime error",
"script_stack": [
"org.elasticsearch.search.lookup.LeafDocLookup.get(LeafDocLookup.java:88)",
"org.elasticsearch.search.lookup.LeafDocLookup.get(LeafDocLookup.java:40)",
"doc['parent_ruuid'].value",
" ^---- HERE"
],
"script": "doc['parent_ruuid'].value",
"lang": "painless",
"position": {
"offset": 4,
"start": 0,
"end": 25
},
"caused_by": {
"type": "illegal_argument_exception",
"reason": "No field found for [parent_ruuid] in mapping with types "
}
}
},
{
"shard": 0,
"index": "snaplogic_2021-06",
"node": "TjZg_rNkQd-OL5wYTsAg1Q",
"reason": {
"type": "script_exception",
"reason": "runtime error",
"script_stack": [
"org.elasticsearch.index.mapper.TextFieldMapper$TextFieldType.fielddataBuilder(TextFieldMapper.java:823)",
"org.elasticsearch.index.fielddata.IndexFieldDataService.getForField(IndexFieldDataService.java:120)",
"org.elasticsearch.index.query.QueryShardContext.lambda$lookup$1(QueryShardContext.java:328)",
"org.elasticsearch.search.lookup.SearchLookup.lambda$new$1(SearchLookup.java:68)",
"org.elasticsearch.search.lookup.LeafDocLookup$1.run(LeafDocLookup.java:95)",
"org.elasticsearch.search.lookup.LeafDocLookup$1.run(LeafDocLookup.java:92)",
"java.base/java.security.AccessController.doPrivileged(AccessController.java:312)",
"org.elasticsearch.search.lookup.LeafDocLookup.get(LeafDocLookup.java:92)",
"org.elasticsearch.search.lookup.LeafDocLookup.get(LeafDocLookup.java:40)",
"doc['parent_ruuid'].value",
" ^---- HERE"
],
"script": "doc['parent_ruuid'].value",
"lang": "painless",
"position": {
"offset": 4,
"start": 0,
"end": 25
},
"caused_by": {
"type": "illegal_argument_exception",
"reason": "Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [parent_ruuid] in order to load field data by uninverting the inverted index. Note that this can use significant memory."
}
}
}
]
},
"hits": {
"total": 982,
"max_score": null,
"hits":
}
}
This must be the reason, I assume that the field you are trying to use with the scripted field is not a keyword
Yes but on your scripted field you do not use the keyword value. I think if you change it to
doc['parent_ruuid.keyword'].value it will not fail
This fixed it though there is a error msg.
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 4,
"successful": 3,
"skipped": 2,
"failed": 1,
"failures": [
{
"shard": 0,
"index": "snaplogic_2021",
"node": "TjZg_rNkQd-OL5wYTsAg1Q",
"reason": {
"type": "script_exception",
"reason": "runtime error",
"script_stack": [
"org.elasticsearch.search.lookup.LeafDocLookup.get(LeafDocLookup.java:88)",
"org.elasticsearch.search.lookup.LeafDocLookup.get(LeafDocLookup.java:40)",
"doc['id.keyword'].value",
" ^---- HERE"
],
"script": "doc['id.keyword'].value",
"lang": "painless",
"position": {
"offset": 4,
"start": 0,
"end": 23
},
"caused_by": {
"type": "illegal_argument_exception",
"reason": "No field found for [id.keyword] in mapping with types "
}
}
}
]
},
"hits": {
"total": 8,
"max_score": null,
"hits":
},
"aggregations": {
"2": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 2,
"buckets": [
{
"key": "5b7548984dd5a8001fb7e9ca_d3ded0f5-6a70-4bce-bfbe-c42869082210",
"doc_count": 2
},
{
"key": "5b7548984dd5a8001fb7e9ca_0082ee34-080b-4e82-8a06-fdcc64ef63b5",
"doc_count": 1
},
{
"key": "5b7548984dd5a8001fb7e9ca_38717610-737e-4983-abe8-ae90fc087785",
"doc_count": 1
},
{
"key": "5b7548984dd5a8001fb7e9ca_3df0c02e-93b4-431d-a7b5-96ec2f15221f",
"doc_count": 1
},
{
"key": "5b7548984dd5a8001fb7e9ca_780a28c2-e186-4c7a-9d12-9c192e7f80ee",
"doc_count": 1
}
]
}
}
}
Does the id.keyword exists? Moreover I saw that you opened this issue Formatting and Scripting fields not working. Not able to combine fields. · Issue #101140 · elastic/kibana · GitHub. We can close it right? 
Yes you can it is the same issue you can close it.
So now it works right?
From what I can see everything is as I wanted it to look like but there is an error.
If you change your script to doc['id'].value does it work?
Nope same error as before, it needs the .keyword
© 2020. All Rights Reserved - Elasticsearch
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.
