Sorting based on a dynamic field array?

Derry_O_Sullivan · January 31, 2013, 4:58pm

Hi all,

I'm trying to sort based on a dynamic field (it's an array of values).

The field is present in the mapping:

Request to :9200/items/item/_mapping:

{

item:
{
- properties:
  {
  - ...,
    ...,
  - selects:
    {
    - dynamic: "true",
    - properties:
      {
      - community_id:
        {
        
        type: "string"
        },
      - num_selects:
        {
        
        type: "long"
        }
        }
        },

I then go to create a search query which sorts by the number of values in
this field (e.g. size of selects) and thought i could do:

customScoreQuery(matchAllQuery())).script("doc['selects'].values.length"));
//not sure if length should be size?

Whether i use length or size, i get:

[Line: 1, Column: 1]]; nested: ElasticSearchIllegalArgumentException[No
field found for [selects] in mapping with types [item]];
}{[2Qze4mSDQiuf16o3GxVP1Q][items][0]:
QueryPhaseExecutionException[[items][0]: query[filtered(custom score
(application_id:1,function=script[doc['selects'].values.size], params
[null]))->cache(_type:item)],from[0],size[10]: Query Failed [Failed to
execute main query]]; nested: CompileException[[Error: No field found for
[selects] in mapping with types [item]]
[Near : {... doc['selects'].values.size ....}]

I thought the issue may be because not all the 'selects' may have values?
But the error seems to suggest that the mapping has an issue?

I'm running ES 0.20.4; there is currently ~630,000 docs that the
matchAllQuery() returns.

Any help greatly appreciated.

Derry

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Derry_O_Sullivan · January 31, 2013, 5:07pm

I've tested this on another field (string array) which is not dynamic and
it works fine....

On Thursday, 31 January 2013 16:58:28 UTC, Derry O' Sullivan wrote:

Hi all,

I'm trying to sort based on a dynamic field (it's an array of values).

The field is present in the mapping:

Request to :9200/items/item/_mapping:

{

item:
{

properties:
{

...,
...,

selects:
{

dynamic: "true",

properties:
{

community_id:
{

type: "string"
},

num_selects:
{

type: "long"
}
}
},

I then go to create a search query which sorts by the number of values in
this field (e.g. size of selects) and thought i could do:

customScoreQuery(matchAllQuery())).script("doc['selects'].values.length"));
//not sure if length should be size?

Whether i use length or size, i get:

[Line: 1, Column: 1]]; nested: ElasticSearchIllegalArgumentException[No
field found for [selects] in mapping with types [item]];
}{[2Qze4mSDQiuf16o3GxVP1Q][items][0]:
QueryPhaseExecutionException[[items][0]: query[filtered(custom score
(application_id:1,function=script[doc['selects'].values.size], params
[null]))->cache(_type:item)],from[0],size[10]: Query Failed [Failed to
execute main query]]; nested: CompileException[[Error: No field found for
[selects] in mapping with types [item]]
[Near : {... doc['selects'].values.size ....}]

I thought the issue may be because not all the 'selects' may have values?
But the error seems to suggest that the mapping has an issue?

I'm running ES 0.20.4; there is currently ~630,000 docs that the
matchAllQuery() returns.

Any help greatly appreciated.

Derry

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Derry_O_Sullivan · February 1, 2013, 12:39pm

I've also tried to do this using a custom scoring script:

$ curl -XGET 'http://localhost:9200/items/item/_search' -d '{
"query" :{
"custom_score": {
"query" : { "match_all": {}},
"script" : "selectsscorescript",
"lang": "native"
}
}
}'
% Total % Received % Xferd Average Speed Time Time Time
Current
Dload Upload Total Spent Left
Speed
100 1033 100 882 100 151 55125 9437 --:--:-- --:--:-- --:--:--
861k{"error":"SearchPhaseExecutionException[Failed to execute phase
[query], total failure; shardFailures {[4OYazvnmR7CZXJVNET_ncA][items][4]:
QueryPhaseExecutionException[[items][4]: query[filtered(custom score
(ConstantScore(NotDeleted(:)),function=script[selectsscorescript], params
[null]))->cache(_type:item)],from[0],size[10]: Query Failed [Failed to
execute main query]]; nested: ElasticSearchIllegalArgumentException[No
field found for [selects] in mapping with types [item]];
}{[4OYazvnmR7CZXJVNET_ncA][items][3]:
QueryPhaseExecutionException[[items][3]: query[filtered(custom score
(ConstantScore(NotDeleted(:)),function=script[selectsscorescript], params
[null]))->cache(_type:item)],from[0],size[10]: Query Failed [Failed to
execute main query]]; nested: ElasticSearchIllegalArgumentException[No
field found for [selects] in mapping with types [item]]; }]","status":500}

Could someone please comment on this? I'm really confused by the fact that
is is saying there is no field found when it is clearly there....

On Thursday, 31 January 2013 17:07:40 UTC, Derry O' Sullivan wrote:

I've tested this on another field (string array) which is not dynamic and
it works fine....

On Thursday, 31 January 2013 16:58:28 UTC, Derry O' Sullivan wrote:

Hi all,

I'm trying to sort based on a dynamic field (it's an array of values).

The field is present in the mapping:

Request to :9200/items/item/_mapping:

{

item:
{

properties:
{

...,
...,

selects:
{

dynamic: "true",

properties:
{

community_id:
{

type: "string"
},

num_selects:
{

type: "long"
}
}
},

I then go to create a search query which sorts by the number of values in
this field (e.g. size of selects) and thought i could do:

customScoreQuery(matchAllQuery())).script("doc['selects'].values.length"));
//not sure if length should be size?

Whether i use length or size, i get:

[Line: 1, Column: 1]]; nested: ElasticSearchIllegalArgumentException[No
field found for [selects] in mapping with types [item]];
}{[2Qze4mSDQiuf16o3GxVP1Q][items][0]:
QueryPhaseExecutionException[[items][0]: query[filtered(custom score
(application_id:1,function=script[doc['selects'].values.size], params
[null]))->cache(_type:item)],from[0],size[10]: Query Failed [Failed to
execute main query]]; nested: CompileException[[Error: No field found for
[selects] in mapping with types [item]]
[Near : {... doc['selects'].values.size ....}]

I thought the issue may be because not all the 'selects' may have values?
But the error seems to suggest that the mapping has an issue?

I'm running ES 0.20.4; there is currently ~630,000 docs that the
matchAllQuery() returns.

Any help greatly appreciated.

Derry

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Derry_O_Sullivan · February 1, 2013, 12:58pm

Discovered the answer for anyone wondering

You have to do a .containsKey() check if you have a potential
null field in a doc.

Also from the script expression doc (linked
here): https://groups.google.com/d/msg/elasticsearch/P0DG9j6hWKY/wGLpg5d9bswJ
you can only use doc() on simple fields!

On Friday, 1 February 2013 12:39:46 UTC, Derry O' Sullivan wrote:

I've also tried to do this using a custom scoring script:

$ curl -XGET 'http://localhost:9200/items/item/_search' -d '{
"query" :{
"custom_score": {
"query" : { "match_all": {}},
"script" : "selectsscorescript",
"lang": "native"
}
}
}'
% Total % Received % Xferd Average Speed Time Time Time
Current
Dload Upload Total Spent Left
Speed
100 1033 100 882 100 151 55125 9437 --:--:-- --:--:-- --:--:--
861k{"error":"SearchPhaseExecutionException[Failed to execute phase
[query], total failure; shardFailures {[4OYazvnmR7CZXJVNET_ncA][items][4]:
QueryPhaseExecutionException[[items][4]: query[filtered(custom score
(ConstantScore(NotDeleted(:)),function=script[selectsscorescript], params
[null]))->cache(_type:item)],from[0],size[10]: Query Failed [Failed to
execute main query]]; nested: ElasticSearchIllegalArgumentException[No
field found for [selects] in mapping with types [item]];
}{[4OYazvnmR7CZXJVNET_ncA][items][3]:
QueryPhaseExecutionException[[items][3]: query[filtered(custom score
(ConstantScore(NotDeleted(:)),function=script[selectsscorescript], params
[null]))->cache(_type:item)],from[0],size[10]: Query Failed [Failed to
execute main query]]; nested: ElasticSearchIllegalArgumentException[No
field found for [selects] in mapping with types [item]]; }]","status":500}

Could someone please comment on this? I'm really confused by the fact that
is is saying there is no field found when it is clearly there....

On Thursday, 31 January 2013 17:07:40 UTC, Derry O' Sullivan wrote:

I've tested this on another field (string array) which is not dynamic and
it works fine....

On Thursday, 31 January 2013 16:58:28 UTC, Derry O' Sullivan wrote:

Hi all,

I'm trying to sort based on a dynamic field (it's an array of values).

The field is present in the mapping:

Request to :9200/items/item/_mapping:

{

item:
{

properties:
{

...,
...,

selects:
{

dynamic: "true",

properties:
{

community_id:
{

type: "string"
},

num_selects:
{

type: "long"
}
}
},

I then go to create a search query which sorts by the number of values
in this field (e.g. size of selects) and thought i could do:

customScoreQuery(matchAllQuery())).script("doc['selects'].values.length"));
//not sure if length should be size?

Whether i use length or size, i get:

[Line: 1, Column: 1]]; nested: ElasticSearchIllegalArgumentException[No
field found for [selects] in mapping with types [item]];
}{[2Qze4mSDQiuf16o3GxVP1Q][items][0]:
QueryPhaseExecutionException[[items][0]: query[filtered(custom score
(application_id:1,function=script[doc['selects'].values.size], params
[null]))->cache(_type:item)],from[0],size[10]: Query Failed [Failed to
execute main query]]; nested: CompileException[[Error: No field found for
[selects] in mapping with types [item]]
[Near : {... doc['selects'].values.size ....}]

I thought the issue may be because not all the 'selects' may have
values? But the error seems to suggest that the mapping has an issue?

I'm running ES 0.20.4; there is currently ~630,000 docs that the
matchAllQuery() returns.

Any help greatly appreciated.

Derry

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Topic		Replies	Views
Sort on "long" type field Elasticsearch	5	3695	July 6, 2017
Searching and sorting dynamic fields Elasticsearch	4	1827	July 5, 2017
Scripting Query for returning cases where array size is larger than X Elasticsearch	3	968	February 11, 2020
Not able to sort on Long field and Function score also not working Elasticsearch	2	220	April 17, 2023
Sort Based on Subfield Omits Results Elasticsearch	1	392	April 2, 2018

Sorting based on a dynamic field array?

Related topics