Sort over dynamic fields

Elastic Stack Elasticsearch
1

Hi,

when i try to sort over a dynamically created field the query crashes when
the field does not exist cause there is no mapping for this field available.

  1. is there a way to set something like a default mapping with wildcards
    for dyn fields like "myid_*" ?
  2. is it possible to combine an exist check with the sorting?
  3. other solutions?

thx & regards
tom

--

2

Answers inline.

On Thu, Jan 24, 2013 at 12:39 AM, Tom t.oppelt@superreal.de wrote:

Hi,

when i try to sort over a dynamically created field the query crashes when
the field does not exist cause there is no mapping for this field available.

  1. is there a way to set something like a default mapping with wildcards
    for dyn fields like "myid_*" ?

You can use dynamic templates (scroll down to dynamic_templates):

  1. is it possible to combine an exist check with the sorting?

Not sure what your use case is, but can you use the Missing Values feature?
You can put documents that do not have the field to the bottom, or give
them a default value that can be filterered out on the client side.

  1. other solutions?

Depends on your use case. What is not working with the existing solutions?

thx & regards
tom

--

Cheers,

Ivan

--

3

Am Donnerstag, 24. Januar 2013 18:02:02 UTC+1 schrieb Ivan Brusic:

Answers inline.

On Thu, Jan 24, 2013 at 12:39 AM, Tom <t.op...@superreal.de <javascript:>>wrote:

Hi,

when i try to sort over a dynamically created field the query crashes
when the field does not exist cause there is no mapping for this field
available.

  1. is there a way to set something like a default mapping with wildcards
    for dyn fields like "myid_*" ?

You can use dynamic templates (scroll down to dynamic_templates):
Elasticsearch Platform — Find real-time answers at scale | Elastic

It didn't work for my sorting problem:
Test Mapping:

curl -XPUT 'http://localhost:9200/test/item/_mapping?pretty=true' -d
'
{
"item" : {
"dynamic_templates" : [
{
"template_1" : {
"match" : "mysort_*",
"mapping" : {
"type" : "integer",
"index" : "not_analyzed",
"store" : "yes"
} } }
],
properties: {
"message" : {
"type" : "string",
"store" : "yes",
"index" : "analyzed"
}
}
}
}
'

Query:

curl -XGET 'http://localhost:9200/test/item/_search?pretty=true' -d
'
{
"sort":[
{ "test_notexist":{ "order":"asc" } }
],
"query":{
(...)
},
"filter":{
"exists":{ "field":"test_notexist" }
}
}
'

This query crashes with:

Parse Failure [No mapping found for [test_notexists] in order to sort on
...

Did i miss something?

--

4

sorry i pasted wrong query in my last post of course i query for
mysort_notexists field. but it makes no difference.

Am Donnerstag, 24. Januar 2013 09:39:45 UTC+1 schrieb Tom:

Hi,

when i try to sort over a dynamically created field the query crashes when
the field does not exist cause there is no mapping for this field available.

  1. is there a way to set something like a default mapping with wildcards
    for dyn fields like "myid_*" ?
  2. is it possible to combine an exist check with the sorting?
  3. other solutions?

thx & regards
tom

--

5

Sorry! I answered your questions, but forgot to address the main issue.

Try using the ignore_unmapped option when sorting against on unmapped
field. Might not be the best solution, but it will stop the exception from
occurring.

--
Ivan

On Fri, Jan 25, 2013 at 6:51 AM, Tom t.oppelt@superreal.de wrote:

sorry i pasted wrong query in my last post of course i query for
mysort_notexists field. but it makes no difference.

Am Donnerstag, 24. Januar 2013 09:39:45 UTC+1 schrieb Tom:

Hi,

when i try to sort over a dynamically created field the query crashes
when the field does not exist cause there is no mapping for this field
available.

  1. is there a way to set something like a default mapping with wildcards
    for dyn fields like "myid_*" ?
  2. is it possible to combine an exist check with the sorting?
  3. other solutions?

thx & regards
tom

--

--

6

hm, ok, but actually this field is mapped by dynamic_template.
so there is a mapping available actually, but the specific dynamic field
does not exist.
disturbing.

Am Freitag, 25. Januar 2013 17:02:52 UTC+1 schrieb Ivan Brusic:

Sorry! I answered your questions, but forgot to address the main issue.

Try using the ignore_unmapped option when sorting against on unmapped
field. Might not be the best solution, but it will stop the exception from
occurring.

--
Ivan

On Fri, Jan 25, 2013 at 6:51 AM, Tom <t.op...@superreal.de <javascript:>>wrote:

sorry i pasted wrong query in my last post of course i query for
mysort_notexists field. but it makes no difference.

Am Donnerstag, 24. Januar 2013 09:39:45 UTC+1 schrieb Tom:

Hi,

when i try to sort over a dynamically created field the query crashes
when the field does not exist cause there is no mapping for this field
available.

  1. is there a way to set something like a default mapping with wildcards
    for dyn fields like "myid_*" ?
  2. is it possible to combine an exist check with the sorting?
  3. other solutions?

thx & regards
tom

--

--