Created script field successfully but error occurred in discover

[Echo_yu]

Hi, I created a script field with the type of date successfully but when I search data in discover something went wrong . And I cannot use my script field in aggregation either. My script field create by if(doc['finished'].value.getHour()>=16) { Instant.ofEpochMilli(doc['finished'].value.getMillis()+8*60*60*1000); } else { Instant.ofEpochMilli(doc['finished'].value.getMillis()); }. And here is the error below:

Detail of exception:
Request to Elasticsearch failed: {"error":{"root_cause":[{"type":"script_exception","reason":"runtime error","script_stack":["org.elasticsearch.index.fielddata.ScriptDocValues$Dates.get(ScriptDocValues.java:160)","org.elasticsearch.index.fielddata.ScriptDocValues$Dates.getValue(ScriptDocValues.java:154)","if(doc['finished'].value.getHour()>=16)\n{\n "," ^---- HERE"],"script":"if(doc['finished'].value.getHour()>=16)\n{\n Instant.ofEpochMilli(doc['finished'].value.getMillis()+8*60*60*1000);\n}\nelse\n{\n Instant.ofEpochMilli(doc['finished'].value.getMillis());\n}","lang":"painless"}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"bp-session-log-b","node":"VlTdaqFERuaqpqtJyFR8Sw","reason":{"type":"script_exception","reason":"runtime error","script_stack":["org.elasticsearch.index.fielddata.ScriptDocValues$Dates.get(ScriptDocValues.java:160)","org.elasticsearch.index.fielddata.ScriptDocValues$Dates.getValue(ScriptDocValues.java:154)","if(doc['finished'].value.getHour()>=16)\n{\n "," ^---- HERE"],"script":"if(doc['finished'].value.getHour()>=16)\n{\n Instant.ofEpochMilli(doc['finished'].value.getMillis()+8*60*60*1000);\n}\nelse\n{\n Instant.ofEpochMilli(doc['finished'].value.getMillis());\n}","lang":"painless","caused_by":{"type":"illegal_state_exception","reason":"A document doesn't have a value for a field! Use doc[<field>].size()==0 to check if a document is missing a field!"}}}]},"status":400}
Looking forward to your help，Thanks！

[dosant (Anton Dosov)]

Hi @Echo_yu,

I haven't tried anything yet, but looking at the error:

@Echo_yu

"reason":"A document doesn't have a value for a field! Use doc[].size()==0 to check if a document is missing a field!"}}}]},

Looks like there is a null pointer exception and some of the docs are missing finished field.
Have you tried recommendation from error to check if the field is present first and then access it?

[Echo_yu]

Hi， I have tried this recommendation and I have already dealt with this problem. However I got a new problem of script field. Why did I aggregate the new script field which was named finial_date in kibana's dev tool but always returned null? My code is：
GET /bp-session-log-all/_search
{
"query": {
"bool": {
"must": [
{"match": {
"workqueue_name.keyword": "STR_WQ_1.2 Process STC"
}},
{"match": {
"txn_success": 0}}]}},
"aggs": {
"datetime": {
"date_range": {
"field": "finial_date",
"ranges": [
{
"from": "now-1d/d",
"to": "now/d"
}]}}}}
Looking forwarding to your help. Thanks!

[dosant (Anton Dosov)]

@Echo_yu, scripted field is created on index pattern level which is part of Kibana, and not Elasticsearch.

If you want to see aggregation with scripted field in dev tools, you'd have to manually duplicate your script inside that aggregation. There is a documented example using a script for a sum aggregation which may help. To avoid duplication, you could also store the script and reference in both locations by name. - How to use a script field inside Aggregate?

But this scripted field should be available for aggregations in Kibana's visualize. So you should be able to create a visualisation on that field. Internally Kibana will use your script inside the aggregation in request to Elasticsearch.

Hope this is helpful

[Echo_yu]

Thanks for your help, I will try your recommendation later