Sorting based on a dynamic field array?

Elastic Stack Elasticsearch
1

Hi all,

I'm trying to sort based on a dynamic field (it's an array of values).

The field is present in the mapping:

Request to :9200/items/item/_mapping:

{

  • item:
    {
    • properties:
      {
      • ...,
        ...,
      • selects:
        {
        • dynamic: "true",
        • properties:
          {
          • community_id:
            {
            • type: "string"
              },
          • num_selects:
            {
            • type: "long"
              }
              }
              },

I then go to create a search query which sorts by the number of values in
this field (e.g. size of selects) and thought i could do:

customScoreQuery(matchAllQuery())).script("doc['selects'].values.length"));
//not sure if length should be size?

Whether i use length or size, i get:

[Line: 1, Column: 1]]; nested: ElasticSearchIllegalArgumentException[No
field found for [selects] in mapping with types [item]];
}{[2Qze4mSDQiuf16o3GxVP1Q][items][0]:
QueryPhaseExecutionException[[items][0]: query[filtered(custom score
(application_id:1,function=script[doc['selects'].values.size], params
[null]))->cache(_type:item)],from[0],size[10]: Query Failed [Failed to
execute main query]]; nested: CompileException[[Error: No field found for
[selects] in mapping with types [item]]
[Near : {... doc['selects'].values.size ....}]

I thought the issue may be because not all the 'selects' may have values?
But the error seems to suggest that the mapping has an issue?

I'm running ES 0.20.4; there is currently ~630,000 docs that the
matchAllQuery() returns.

Any help greatly appreciated.

Derry

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

I've tested this on another field (string array) which is not dynamic and
it works fine....

On Thursday, 31 January 2013 16:58:28 UTC, Derry O' Sullivan wrote:

Hi all,

I'm trying to sort based on a dynamic field (it's an array of values).

The field is present in the mapping:

Request to :9200/items/item/_mapping:

{

  • item:
    {
    • properties:
      {
      • ...,
        ...,
      • selects:
        {
        • dynamic: "true",
        • properties:
          {
          • community_id:
            {
            • type: "string"
              },
          • num_selects:
            {
            • type: "long"
              }
              }
              },

I then go to create a search query which sorts by the number of values in
this field (e.g. size of selects) and thought i could do:

customScoreQuery(matchAllQuery())).script("doc['selects'].values.length"));
//not sure if length should be size?

Whether i use length or size, i get:

[Line: 1, Column: 1]]; nested: ElasticSearchIllegalArgumentException[No
field found for [selects] in mapping with types [item]];
}{[2Qze4mSDQiuf16o3GxVP1Q][items][0]:
QueryPhaseExecutionException[[items][0]: query[filtered(custom score
(application_id:1,function=script[doc['selects'].values.size], params
[null]))->cache(_type:item)],from[0],size[10]: Query Failed [Failed to
execute main query]]; nested: CompileException[[Error: No field found for
[selects] in mapping with types [item]]
[Near : {... doc['selects'].values.size ....}]

I thought the issue may be because not all the 'selects' may have values?
But the error seems to suggest that the mapping has an issue?

I'm running ES 0.20.4; there is currently ~630,000 docs that the
matchAllQuery() returns.

Any help greatly appreciated.

Derry

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

I've also tried to do this using a custom scoring script:

$ curl -XGET 'http://localhost:9200/items/item/_search' -d '{
"query" :{
"custom_score": {
"query" : { "match_all": {}},
"script" : "selectsscorescript",
"lang": "native"
}
}
}'
% Total % Received % Xferd Average Speed Time Time Time
Current
Dload Upload Total Spent Left
Speed
100 1033 100 882 100 151 55125 9437 --:--:-- --:--:-- --:--:--
861k{"error":"SearchPhaseExecutionException[Failed to execute phase
[query], total failure; shardFailures {[4OYazvnmR7CZXJVNET_ncA][items][4]:
QueryPhaseExecutionException[[items][4]: query[filtered(custom score
(ConstantScore(NotDeleted(:)),function=script[selectsscorescript], params
[null]))->cache(_type:item)],from[0],size[10]: Query Failed [Failed to
execute main query]]; nested: ElasticSearchIllegalArgumentException[No
field found for [selects] in mapping with types [item]];
}{[4OYazvnmR7CZXJVNET_ncA][items][3]:
QueryPhaseExecutionException[[items][3]: query[filtered(custom score
(ConstantScore(NotDeleted(:)),function=script[selectsscorescript], params
[null]))->cache(_type:item)],from[0],size[10]: Query Failed [Failed to
execute main query]]; nested: ElasticSearchIllegalArgumentException[No
field found for [selects] in mapping with types [item]]; }]","status":500}

Could someone please comment on this? I'm really confused by the fact that
is is saying there is no field found when it is clearly there....

On Thursday, 31 January 2013 17:07:40 UTC, Derry O' Sullivan wrote:

I've tested this on another field (string array) which is not dynamic and
it works fine....

On Thursday, 31 January 2013 16:58:28 UTC, Derry O' Sullivan wrote:

Hi all,

I'm trying to sort based on a dynamic field (it's an array of values).

The field is present in the mapping:

Request to :9200/items/item/_mapping:

{

  • item:
    {
    • properties:
      {
      • ...,
        ...,
      • selects:
        {
        • dynamic: "true",
        • properties:
          {
          • community_id:
            {
            • type: "string"
              },
          • num_selects:
            {
            • type: "long"
              }
              }
              },

I then go to create a search query which sorts by the number of values in
this field (e.g. size of selects) and thought i could do:

customScoreQuery(matchAllQuery())).script("doc['selects'].values.length"));
//not sure if length should be size?

Whether i use length or size, i get:

[Line: 1, Column: 1]]; nested: ElasticSearchIllegalArgumentException[No
field found for [selects] in mapping with types [item]];
}{[2Qze4mSDQiuf16o3GxVP1Q][items][0]:
QueryPhaseExecutionException[[items][0]: query[filtered(custom score
(application_id:1,function=script[doc['selects'].values.size], params
[null]))->cache(_type:item)],from[0],size[10]: Query Failed [Failed to
execute main query]]; nested: CompileException[[Error: No field found for
[selects] in mapping with types [item]]
[Near : {... doc['selects'].values.size ....}]

I thought the issue may be because not all the 'selects' may have values?
But the error seems to suggest that the mapping has an issue?

I'm running ES 0.20.4; there is currently ~630,000 docs that the
matchAllQuery() returns.

Any help greatly appreciated.

Derry

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

4

Discovered the answer for anyone wondering

You have to do a .containsKey() check if you have a potential
null field in a doc.

Also from the script expression doc (linked
here): https://groups.google.com/d/msg/elasticsearch/P0DG9j6hWKY/wGLpg5d9bswJ
you can only use doc() on simple fields!

On Friday, 1 February 2013 12:39:46 UTC, Derry O' Sullivan wrote:

I've also tried to do this using a custom scoring script:

$ curl -XGET 'http://localhost:9200/items/item/_search' -d '{
"query" :{
"custom_score": {
"query" : { "match_all": {}},
"script" : "selectsscorescript",
"lang": "native"
}
}
}'
% Total % Received % Xferd Average Speed Time Time Time
Current
Dload Upload Total Spent Left
Speed
100 1033 100 882 100 151 55125 9437 --:--:-- --:--:-- --:--:--
861k{"error":"SearchPhaseExecutionException[Failed to execute phase
[query], total failure; shardFailures {[4OYazvnmR7CZXJVNET_ncA][items][4]:
QueryPhaseExecutionException[[items][4]: query[filtered(custom score
(ConstantScore(NotDeleted(:)),function=script[selectsscorescript], params
[null]))->cache(_type:item)],from[0],size[10]: Query Failed [Failed to
execute main query]]; nested: ElasticSearchIllegalArgumentException[No
field found for [selects] in mapping with types [item]];
}{[4OYazvnmR7CZXJVNET_ncA][items][3]:
QueryPhaseExecutionException[[items][3]: query[filtered(custom score
(ConstantScore(NotDeleted(:)),function=script[selectsscorescript], params
[null]))->cache(_type:item)],from[0],size[10]: Query Failed [Failed to
execute main query]]; nested: ElasticSearchIllegalArgumentException[No
field found for [selects] in mapping with types [item]]; }]","status":500}

Could someone please comment on this? I'm really confused by the fact that
is is saying there is no field found when it is clearly there....

On Thursday, 31 January 2013 17:07:40 UTC, Derry O' Sullivan wrote:

I've tested this on another field (string array) which is not dynamic and
it works fine....

On Thursday, 31 January 2013 16:58:28 UTC, Derry O' Sullivan wrote:

Hi all,

I'm trying to sort based on a dynamic field (it's an array of values).

The field is present in the mapping:

Request to :9200/items/item/_mapping:

{

  • item:
    {
    • properties:
      {
      • ...,
        ...,
      • selects:
        {
        • dynamic: "true",
        • properties:
          {
          • community_id:
            {
            • type: "string"
              },
          • num_selects:
            {
            • type: "long"
              }
              }
              },

I then go to create a search query which sorts by the number of values
in this field (e.g. size of selects) and thought i could do:

customScoreQuery(matchAllQuery())).script("doc['selects'].values.length"));
//not sure if length should be size?

Whether i use length or size, i get:

[Line: 1, Column: 1]]; nested: ElasticSearchIllegalArgumentException[No
field found for [selects] in mapping with types [item]];
}{[2Qze4mSDQiuf16o3GxVP1Q][items][0]:
QueryPhaseExecutionException[[items][0]: query[filtered(custom score
(application_id:1,function=script[doc['selects'].values.size], params
[null]))->cache(_type:item)],from[0],size[10]: Query Failed [Failed to
execute main query]]; nested: CompileException[[Error: No field found for
[selects] in mapping with types [item]]
[Near : {... doc['selects'].values.size ....}]

I thought the issue may be because not all the 'selects' may have
values? But the error seems to suggest that the mapping has an issue?

I'm running ES 0.20.4; there is currently ~630,000 docs that the
matchAllQuery() returns.

Any help greatly appreciated.

Derry

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.