I am new to Kibana. I am facing a strange issue. I have an index which has date propety as well as timestamp. When I am trying to extract the date and month from the date property in scripted field , I am getting exception . But When I use time stamp I can see the data coming up in the index.
If I use the below one
def Daymonth =doc['applicationData.transactionTimeStatistics.sessionStartTime'].value.monthOfYear ;
I get this error
But If I use
def Daymonth = doc['timeStamp'].value.year.toString() + doc['timeStamp'].value.monthOfYear.toString() + doc['timeStamp'].value.dayOfMonth ;
I can see the data coming . I cant understand what is issue .
I checked the index . Both are date type properties
tsullivan
February 25, 2020, 12:31am
2
Hi, can you show the mapping of your index, and include the entire script for this scripted field?
Hi sullivan
Below is the scripted field I was trying to create , which gives me error.
doc['applicationData.intiatingPassengerRawMrz.keyword'] + doc['applicationData.kioskPort.keyword'].toString() + doc['applicationData.transactionTimeStatistics.sessionStartTime'].value.year.toString() + doc['applicationData.transactionTimeStatistics.sessionStartTime'].value.monthOfYear.toString() + doc['applicationData.transactionTimeStatistics.sessionStartTime'].value.dayOfMonth
Here is the index. I cant post the entire one as it is big to post here . I will just extract the mapping for the problematic area.
transactionTimeStatistics" : {
"properties" : {
"airportDetailsTime" : {
"type" : "float"
},
"amadeusDcsTime" : {
"type" : "float"
},
"amadeusInternalTime" : {
"type" : "float"
},
"amadeusTime" : {
"type" : "float"
},
"csaAccessControlTime" : {
"type" : "float"
},
"csaSessionEndTime" : {
"type" : "date"
},
"csaSessionStartTime" : {
"type" : "date"
},
"csaTotalTime" : {
"type" : "float"
},
"frequentFlyerTime" : {
"type" : "float"
},
"lastScreenDisplayed" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"lastScreenStartTime" : {
"type" : "date"
},
"machineTime" : {
"type" : "float"
},
"nameMatcherTime" : {
"type" : "float"
},
"paxTime" : {
"type" : "float"
},
"sessionEndTime" : {
"type" : "date"
},
"sessionStartTime" : {
"type" : "date"
},
"subscriptionServiceTime" : {
"type" : "float"
},
"totalTime" : {
"type" : "float"
}
}
}
tsullivan
March 9, 2020, 7:51pm
6
Hi, can you try this script?
doc['applicationData.intiatingPassengerRawMrz.keyword'].value +
doc['applicationData.kioskPort.keyword'].value +
doc['applicationData.transactionTimeStatistics.sessionStartTime'].value.year +
doc['applicationData.transactionTimeStatistics.sessionStartTime'].value.monthOfYear +
doc['applicationData.transactionTimeStatistics.sessionStartTime'].value.dayOfMonth
Hi Sullivan
I tried the the script given by you . I got same error.
{
"root_cause": [
{
"type": "script_exception",
"reason": "runtime error",
"script_stack": [
"org.elasticsearch.index.fielddata.ScriptDocValues$Strings.get(ScriptDocValues.java:496)",
"org.elasticsearch.index.fielddata.ScriptDocValues$Strings.getValue(ScriptDocValues.java:503)",
"doc['applicationData.intiatingPassengerRawMrz.keyword'].value +\n\ndoc['applicationData.kioskPort.keyword'].value +\n\ndoc['applicationData.transactionTimeStatistics.sessionStartTime'].value.year +\n\ndoc['applicationData.transactionTimeStatistics.sessionStartTime'].value.monthOfYear +\n\ndoc['applicationData.transactionTimeStatistics.sessionStartTime'].value.dayOfMonth\n",
" ^---- HERE"
],
"script": "doc['applicationData.intiatingPassengerRawMrz.keyword'].value +\n\ndoc['applicationData.kioskPort.keyword'].value +\n\ndoc['applicationData.transactionTimeStatistics.sessionStartTime'].value.year +\n\ndoc['applicationData.transactionTimeStatistics.sessionStartTime'].value.monthOfYear +\n\ndoc['applicationData.transactionTimeStatistics.sessionStartTime'].value.dayOfMonth\n",
"lang": "painless"
}
],
"type": "search_phase_execution_exception",
"reason": "all shards failed",
"phase": "query",
"grouped": true,
"failed_shards": [
{
"shard": 0,
"index": "report-kiosk-session-",
"node": "zjRc5DLTROSI4UG211ktpg",
"reason": {
"type": "script_exception",
"reason": "runtime error",
"script_stack": [
"org.elasticsearch.index.fielddata.ScriptDocValues$Strings.get(ScriptDocValues.java:496)",
"org.elasticsearch.index.fielddata.ScriptDocValues$Strings.getValue(ScriptDocValues.java:503)",
"doc['applicationData.intiatingPassengerRawMrz.keyword'].value +\n\ndoc['applicationData.kioskPort.keyword'].value +\n\ndoc['applicationData.transactionTimeStatistics.sessionStartTime'].value.year +\n\ndoc['applicationData.transactionTimeStatistics.sessionStartTime'].value.monthOfYear +\n\ndoc['applicationData.transactionTimeStatistics.sessionStartTime'].value.dayOfMonth\n",
" ^---- HERE"
],
"script": "doc['applicationData.intiatingPassengerRawMrz.keyword'].value +\n\ndoc['applicationData.kioskPort.keyword'].value +\n\ndoc['applicationData.transactionTimeStatistics.sessionStartTime'].value.year +\n\ndoc['applicationData.transactionTimeStatistics.sessionStartTime'].value.monthOfYear +\n\ndoc['applicationData.transactionTimeStatistics.sessionStartTime'].value.dayOfMonth\n",
"lang": "painless",
"caused_by": {
"type": "illegal_state_exception",
"reason": "A document doesn't have a value for a field! Use doc[<field>].size()==0 to check if a document is missing a field!"
}
}
}
]
}
tsullivan
March 10, 2020, 4:40pm
8
I see. Well thanks, it looks like the error message is making a suggestion towards the solution. It seems the applicationData.intiatingPassengerRawMrz.keyword does not have a value in all of the documents, which causes the script to fail.
You can try adding a check for these empty values in the script to avoid the failure. Here's a quick example:
if (doc['applicationData.intiatingPassengerRawMrz.keyword'].size() == 0) {
return 'invalid Mrz'
};
Hi sullivan
I tried all possibilities before reaching out to kibana. Even the above check fails.
{
"root_cause": [
{
"type": "script_exception",
"reason": "compile error",
"script_stack": [
"... return 'invalid Mrz' ;\n\n};\n",
" ^---- HERE"
],
"script": "if (doc['applicationData.intiatingPassengerRawMrz.keyword'].size() == 0) {\n\n return 'invalid Mrz' ;\n\n};\n",
"lang": "painless"
}
],
"type": "search_phase_execution_exception",
"reason": "all shards failed",
"phase": "query",
"grouped": true,
"failed_shards": [
{
"shard": 0,
"index": "report-kiosk-session-",
"node": "zjRc5DLTROSI4UG211ktpg",
"reason": {
"type": "script_exception",
"reason": "compile error",
"script_stack": [
"... return 'invalid Mrz' ;\n\n};\n",
" ^---- HERE"
],
"script": "if (doc['applicationData.intiatingPassengerRawMrz.keyword'].size() == 0) {\n\n return 'invalid Mrz' ;\n\n};\n",
"lang": "painless",
"caused_by": {
"type": "illegal_argument_exception",
"reason": "unexpected token [';'] was expecting one of [<EOF>]."
}
}
}
]
}
system
April 7, 2020, 10:08pm
10
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
"type": "script_exception",