Field [] used in expression does not exist in mappings, even after creating…?

Elastic Stack Kibana
1

Hi,

I am very new to both Elasticsearch and Kibana, which I am using for a current project. I have data coming in from an external source, and one of the fields I have in Discover of Kibana is 'clearancetime'. This field is either empty or has a timestamp in the following format:

January 22nd 2017, 00:00:00.000

I have scripted a field which uses the clearance time in a ternary statement. I think get an error in Discover that the field cleartime used in the expression does not exist in mappings. So I tried adding it via sense with the following PUT request:

PUT source*/_mappings/clearancetime
{
   "properties": {
      "data": {
          "type": "date"
      }
   }
}

I get back the response : { "acknowledge": true }

But I still get the same field does not exist in mapping error.

Any ideas?

ADDITIONAL ERROR INFORMATION: (sorry about the format)....

Error: Request to Elasticsearch failed: {"error":
{"root_cause":[{"type":"script_exception",
"reason":"Field [clearancetime] used in expression does not exist in mappings"}],
"type":"search_phase_execution_exception","reason":"all shards failed",
"phase":"query","grouped":true,"failed_shards":
[{"shard:0,"index":"source-raw-2017.01","node":"dewddasOSada_0vJWA",
"reason":{"type":"script_exception","reason":"Error during search with inline script 
[doc['clearancetime'].value > 0 ? doc['clearancetime'].value - doc['initialtime'].value : 0] using lang [expression]",
"caused_by":{"type":"script_exception",
"reason":"Field [clearancetime] used in expression does not exist in mappings"}}}]}}

The scripted field called "duration" has the following ternary expression:

doc['clearancetime'].value > 0 ? doc['clearancetime'].value - doc['initialtime'].value : 0
2

Hi @informatico,

by executing the PUT request you described you are adding a mapping for the document type clearancetime to indices matching source* with one field called data of type date.

I guess what you intend to do is add a mapping for the document type (replace ${YOURDOCTYPE} with the proper value) containing the fields clearancetime and initialtime:

PUT source*/_mappings/${YOURDOCTYPE}
{
   "properties": {
      "clearancetime": {
          "type": "date"
      },
      "initialtime": {
          "type": "date"
      }
   }
}
3

This worked! Thank you so much for your response.

This is a little off topic, but do you know how I can get the value in the duration field in a more informative format such as... 00:00:00. Currenlty I am getting a single long value in milliseconds.

I have tried adding
"format": "00:00:00"
under the type properties in the mapping... however this does not change anything.

I am using .value in the expression of my scripted fields, according to the documentaton this returns a value in milliseconds. So what are the alternatives?

Thanks.

4

You can customize the formatting of fields in Kibana using field formatters. It sounds like the Duration formatter could achieve what you're looking for.

5

Thanks for the response.

I should have mentioned that I'm restricted to Kibana 4.6.

Do you know of any alternatives in this case?

6

That is too bad. I would definitely recommend to update to Version 5 of the Elastic Stack if at all possible.

In the meantime you might be able to make due with a scripted field. Just keep in mind that these fields cannot be queried and can have a negative performance impact.