Using scripted fields in filter range sub-aggregation

Elastic Stack Kibana
1

I'm making a table in Kibana and one of the things I want to be able to show is the count of documents that have the field "outcome" with a value "OVERRIDE" and a date range between some date and now.
The outcome filter is fine; I just use outcome:OVERRIDE, but the date range filter is tripping me up. I've tried using the following format:

date_time:[successful_date_time TO now]

However this doesn't work even though I've specified successful_date_time (which is a scripted field) to have the format "YYYY-MM-DDTHH:mm:ss.SSS" as the moment.js format pattern. An example output from this field would be 2017-10-17T15:52:12.647 , and when I use this value manually for the starting point of the range, i.e. date_time:[2017-10-17T15:52:12.647 TO now], the query works fine, even though successful_date_time outputs this value. Is there a way to use this scripted field as a range marker for my query?

2

Hi Andrew,

sorry for the late reply.

Unfortunately it is not possible to use scripted fields within a query, due to the way Elasticsearch handles those.
With Kibana 6 we are introducing a new query language called Kuery, in which we hope, that we will be able to address that issue in a future version, so it's not as confusing anymore as it is today.

Cheers,
Tim

3

Hi Tim,

Thanks for getting back to me. Just for future reference, does this also mean that regular stored fields also can't be used withing queries? e.g. If I wanted a query like date_time:[@timestamp TO now], would that simply not work or is there a specific syntax that would make it work?

Thanks for the help,

Andrew.

4

Hi Andrew,

sorry my answer was a bit confusing in that! No you cannot reference other fields in queries in general. I meant, that you are also not able to properly query scripted fields at all.

You cannot reference fields, due to the way Elasticsearch stores and query the data internally. It builds up inverse indexes for each field, so at query time, when you query for field x it has no link to any other field value in the same document.

Cheers,
Tim

5

Hi Tim,

Thanks for the quick response. I'll be sure to keep up with Kuery and all of its features!

Kind regards,
Andrew.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.