NullPointerException when using script based sorting from Python client

Elastic Stack Elasticsearch
1

Hi, I am trying to use the script based sorting in my queries. This works
fine when using Marvel Sense to submit the query but with the Elasticsearch
python client I am getting a NullPointerException.

Here is the simplified query in Sense:

GET /igo/task/_search
{
"query": {
"match_all": {}
},
"sort" : [
{"_script": {
"lang": "groovy",
"script": "doc['shot.org']",
"type": "string",
"order":"asc"
}}
]
}

And here is the Python version:

from elasticsearch import Elasticsearch
es = Elasticsearch(["argentina", "peru", "elsalvador"])
es.search(index="igo", doc_type="task", body={"query":{"match_all":{}},
"sort":{"_script":{"order":"asc","type":"string","script":"doc['shot.org'].value"}}
})

Results in :

Traceback (most recent call last):
File "", line 1, in
File
"/rel/lang/python/2.7.3-3/opt-debug-iccMaya2014_64/lib/python2.7/site-packages/elasticsearch/client/utils.py",
line 68, in _wrapped
return func(*args, params=params, **kwargs)
File
"/rel/lang/python/2.7.3-3/opt-debug-iccMaya2014_64/lib/python2.7/site-packages/elasticsearch/client/init.py",
line 440, in search
params=params, body=body)
File
"/rel/lang/python/2.7.3-3/opt-debug-iccMaya2014_64/lib/python2.7/site-packages/elasticsearch/transport.py",
line 276, in perform_request
status, headers, data = connection.perform_request(method, url, params,
body, ignore=ignore, timeout=timeout)
File
"/rel/lang/python/2.7.3-3/opt-debug-iccMaya2014_64/lib/python2.7/site-packages/elasticsearch/connection/http_urllib3.py",
line 55, in perform_request
self._raise_error(response.status, raw_data)
File
"/rel/lang/python/2.7.3-3/opt-debug-iccMaya2014_64/lib/python2.7/site-packages/elasticsearch/connection/base.py",
line 97, in _raise_error
raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code,
error_message, additional_info)
<class 'elasticsearch.exceptions.TransportError'>: TransportError(500,
u'SearchPhaseExecutionException[Failed to execute phase [query], all shards
failed; shardFailures {[MNLGv6W8Qfu1G1b51Bx_xA][igo][0]:
QueryPhaseExecutionException[[igo][0]:
query[ConstantScore(cache(_type:task))],from[0],size[10],sort[<custom:"_script":
org.elasticsearch.search.sort.ScriptSortParser$1@51179aca>]: Query Failed
[Failed to execute main query]]; nested: NullPointerException;
}{[MNLGv6W8Qfu1G1b51Bx_xA][igo][1]: QueryPhaseExecutionException[[igo][1]:
query[ConstantScore(cache(_type:task))],from[0],size[10],sort[<custom:"_script":
org.elasticsearch.search.sort.ScriptSortParser$1@60dd6408>]: Query Failed
[Failed to execute main query]]; nested: NullPointerException;
}{[84YFiTW2RuCkGWu_gILm7g][igo][2]:
RemoteTransportException[[Noh-Varr][inet[/10.110.49.5:9300]][indices:data/read/search[phase/query]]];
nested: QueryPhaseExecutionException[[igo][2]:
query[ConstantScore(cache(_type:task))],from[0],size[10],sort[<custom:"_script":
org.elasticsearch.search.sort.ScriptSortParser$1@16e97194>]: Query Failed
[Failed to execute main query]]; nested: NullPointerException;
}{[84YFiTW2RuCkGWu_gILm7g][igo][3]:
RemoteTransportException[[Noh-Varr][inet[/10.110.49.5:9300]][indices:data/read/search[phase/query]]];
nested: QueryPhaseExecutionException[[igo][3]:
query[ConstantScore(cache(_type:task))],from[0],size[10],sort[<custom:"_script":
org.elasticsearch.search.sort.ScriptSortParser$1@6dcca705>]: Query Failed
[Failed to execute main query]]; nested: NullPointerException;
}{[0_CpAuqRR2aDfbDxa9MXqQ][igo][4]:
RemoteTransportException[[Torso][inet[/10.110.49.10:9300]][indices:data/read/search[phase/query]]];
nested: QueryPhaseExecutionException[[igo][4]:
query[ConstantScore(cache(_type:task))],from[0],size[10],sort[<custom:"_script":
org.elasticsearch.search.sort.ScriptSortParser$1@2da08f40>]: Query Failed
[Failed to execute main query]]; nested: NullPointerException; }]')

Here is the cluster info:

curl -XGET 'peru:9200'
{
"status" : 200,
"name" : "Spider Doppelganger",
"cluster_name" : "dhandoff",
"version" : {
"number" : "1.4.0",
"build_hash" : "bc94bd81298f81c656893ab1ddddd30a99356066",
"build_timestamp" : "2014-11-05T14:26:12Z",
"build_snapshot" : false,
"lucene_version" : "4.10.2"
},
"tagline" : "You Know, for Search"
}

Thanks.
-Selim

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d8c73379-6267-4ec6-a2e9-ee052fd2a5ff%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

Actually I fixed my query in Sense, and now I am getting the same error:

GET /igo/task/_search
{
"query": {
"match_all": {}
},
"sort" : [
{"_script": {
"lang": "groovy",
"script": "doc['shot.org'].value",
"type": "string",
"order":"asc"
}}
]
}

Any ideas why? All documents should have the field "shot.org" for this doc
type.

-Selim
On Wednesday, February 4, 2015 at 3:50:40 PM UTC-8, Selim Tuvi wrote:

Hi, I am trying to use the script based sorting in my queries. This works
fine when using Marvel Sense to submit the query but with the Elasticsearch
python client I am getting a NullPointerException.

Here is the simplified query in Sense:

GET /igo/task/_search
{
"query": {
"match_all": {}
},
"sort" : [
{"_script": {
"lang": "groovy",
"script": "doc['shot.org']",
"type": "string",
"order":"asc"
}}
]
}

And here is the Python version:

from elasticsearch import Elasticsearch
es = Elasticsearch(["argentina", "peru", "elsalvador"])
es.search(index="igo", doc_type="task",
body={"query":{"match_all":{}},
"sort":{"_script":{"order":"asc","type":"string","script":"doc['shot.org'].value"}}
})

Results in :

Traceback (most recent call last):
File "", line 1, in
File
"/rel/lang/python/2.7.3-3/opt-debug-iccMaya2014_64/lib/python2.7/site-packages/elasticsearch/client/utils.py",
line 68, in _wrapped
return func(*args, params=params, **kwargs)
File
"/rel/lang/python/2.7.3-3/opt-debug-iccMaya2014_64/lib/python2.7/site-packages/elasticsearch/client/init.py",
line 440, in search
params=params, body=body)
File
"/rel/lang/python/2.7.3-3/opt-debug-iccMaya2014_64/lib/python2.7/site-packages/elasticsearch/transport.py",
line 276, in perform_request
status, headers, data = connection.perform_request(method, url,
params, body, ignore=ignore, timeout=timeout)
File
"/rel/lang/python/2.7.3-3/opt-debug-iccMaya2014_64/lib/python2.7/site-packages/elasticsearch/connection/http_urllib3.py",
line 55, in perform_request
self._raise_error(response.status, raw_data)
File
"/rel/lang/python/2.7.3-3/opt-debug-iccMaya2014_64/lib/python2.7/site-packages/elasticsearch/connection/base.py",
line 97, in _raise_error
raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code,
error_message, additional_info)
<class 'elasticsearch.exceptions.TransportError'>: TransportError(500,
u'SearchPhaseExecutionException[Failed to execute phase [query], all shards
failed; shardFailures {[MNLGv6W8Qfu1G1b51Bx_xA][igo][0]:
QueryPhaseExecutionException[[igo][0]:
query[ConstantScore(cache(_type:task))],from[0],size[10],sort[<custom:"_script":
org.elasticsearch.search.sort.ScriptSortParser$1@51179aca>]: Query Failed
[Failed to execute main query]]; nested: NullPointerException;
}{[MNLGv6W8Qfu1G1b51Bx_xA][igo][1]: QueryPhaseExecutionException[[igo][1]:
query[ConstantScore(cache(_type:task))],from[0],size[10],sort[<custom:"_script":
org.elasticsearch.search.sort.ScriptSortParser$1@60dd6408>]: Query Failed
[Failed to execute main query]]; nested: NullPointerException;
}{[84YFiTW2RuCkGWu_gILm7g][igo][2]:
RemoteTransportException[[Noh-Varr][inet[/10.110.49.5:9300]][indices:data/read/search[phase/query]]];
nested: QueryPhaseExecutionException[[igo][2]:
query[ConstantScore(cache(_type:task))],from[0],size[10],sort[<custom:"_script":
org.elasticsearch.search.sort.ScriptSortParser$1@16e97194>]: Query Failed
[Failed to execute main query]]; nested: NullPointerException;
}{[84YFiTW2RuCkGWu_gILm7g][igo][3]:
RemoteTransportException[[Noh-Varr][inet[/10.110.49.5:9300]][indices:data/read/search[phase/query]]];
nested: QueryPhaseExecutionException[[igo][3]:
query[ConstantScore(cache(_type:task))],from[0],size[10],sort[<custom:"_script":
org.elasticsearch.search.sort.ScriptSortParser$1@6dcca705>]: Query Failed
[Failed to execute main query]]; nested: NullPointerException;
}{[0_CpAuqRR2aDfbDxa9MXqQ][igo][4]:
RemoteTransportException[[Torso][inet[/10.110.49.10:9300]][indices:data/read/search[phase/query]]];
nested: QueryPhaseExecutionException[[igo][4]:
query[ConstantScore(cache(_type:task))],from[0],size[10],sort[<custom:"_script":
org.elasticsearch.search.sort.ScriptSortParser$1@2da08f40>]: Query Failed
[Failed to execute main query]]; nested: NullPointerException; }]')

Here is the cluster info:

curl -XGET 'peru:9200'
{
"status" : 200,
"name" : "Spider Doppelganger",
"cluster_name" : "dhandoff",
"version" : {
"number" : "1.4.0",
"build_hash" : "bc94bd81298f81c656893ab1ddddd30a99356066",
"build_timestamp" : "2014-11-05T14:26:12Z",
"build_snapshot" : false,
"lucene_version" : "4.10.2"
},
"tagline" : "You Know, for Search"
}

Thanks.
-Selim

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/710f4223-70fe-449b-943f-b9445fa7db80%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

3

Actually I am wrong. Looks like some documents did not get created with the
shot.org field. I am just surprised that the sort script syntax is more
sensitive compared to the sort field syntax. Is there a way to handle this
inside the groovy script? I tried checking for null inside the script and
not returning anything but that didn't help.

Thanks
-Selim

On Wednesday, February 4, 2015 at 3:57:00 PM UTC-8, Selim Tuvi wrote:

Actually I fixed my query in Sense, and now I am getting the same error:

GET /igo/task/_search
{
"query": {
"match_all": {}
},
"sort" : [
{"_script": {
"lang": "groovy",
"script": "doc['shot.org'].value",
"type": "string",
"order":"asc"
}}
]
}

Any ideas why? All documents should have the field "shot.org" for this
doc type.

-Selim

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5f5d18dc-f4fc-4895-a037-95ddf6613fab%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

4

Hi,

Something like this should work:

"doc['shot.org'] ? doc[’shot.org'].value : ’default_value'"

Masaru

On February 5, 2015 at 10:23:34, 'Selim Tuvi' via elasticsearch (elasticsearch@googlegroups.com) wrote:

Actually I am wrong. Looks like some documents did not get created with the
shot.org field. I am just surprised that the sort script syntax is more
sensitive compared to the sort field syntax. Is there a way to handle this
inside the groovy script? I tried checking for null inside the script and
not returning anything but that didn't help.

Thanks
-Selim

On Wednesday, February 4, 2015 at 3:57:00 PM UTC-8, Selim Tuvi wrote:

Actually I fixed my query in Sense, and now I am getting the same error:

GET /igo/task/_search
{
"query": {
"match_all": {}
},
"sort" : [
{"_script": {
"lang": "groovy",
"script": "doc['shot.org'].value",
"type": "string",
"order":"asc"
}}
]
}

Any ideas why? All documents should have the field "shot.org" for this
doc type.

-Selim

--
You received this message because you are subscribed to the Google Groups "elasticsearch"
group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5f5d18dc-f4fc-4895-a037-95ddf6613fab%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.54d2d11a.12200854.1c2%40citra.local.
For more options, visit https://groups.google.com/d/optout.

5

Do you have a full stack trace in server logs?

David

Le 5 févr. 2015 à 03:09, Masaru Hasegawa haniomasali@gmail.com a écrit :

Hi,

Something like this should work:

"doc['shot.org'] ? doc[’shot.org'].value : ’default_value'"

Masaru

On February 5, 2015 at 10:23:34, 'Selim Tuvi' via elasticsearch (elasticsearch@googlegroups.com) wrote:
Actually I am wrong. Looks like some documents did not get created with the
shot.org field. I am just surprised that the sort script syntax is more
sensitive compared to the sort field syntax. Is there a way to handle this
inside the groovy script? I tried checking for null inside the script and
not returning anything but that didn't help.

Thanks
-Selim

On Wednesday, February 4, 2015 at 3:57:00 PM UTC-8, Selim Tuvi wrote:

Actually I fixed my query in Sense, and now I am getting the same error:

GET /igo/task/_search
{
"query": {
"match_all": {}
},
"sort" : [
{"_script": {
"lang": "groovy",
"script": "doc['shot.org'].value",
"type": "string",
"order":"asc"
}}
]
}

Any ideas why? All documents should have the field "shot.org" for this
doc type.

-Selim

--
You received this message because you are subscribed to the Google Groups "elasticsearch"
group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5f5d18dc-f4fc-4895-a037-95ddf6613fab%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.54d2d11a.12200854.1c2%40citra.local.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/7535A4AB-C403-4EB5-B810-C7B0DFBB5D09%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.

6

Thanks Masaru, I ended up coding this and that worked:

shot = doc['shot.org'].value; if (shot==null) return '';
.
.

-Selim

On Wednesday, February 4, 2015 at 6:10:48 PM UTC-8, Masaru Hasegawa wrote:

Hi,

Something like this should work:

"doc['shot.org'] ? doc[’shot.org'].value : ’default_value'"

Masaru

On February 5, 2015 at 10:23:34, 'Selim Tuvi' via elasticsearch (
elasti...@googlegroups.com <javascript:>) wrote:

Actually I am wrong. Looks like some documents did not get created with
the
shot.org field. I am just surprised that the sort script syntax is more
sensitive compared to the sort field syntax. Is there a way to handle
this
inside the groovy script? I tried checking for null inside the script
and
not returning anything but that didn't help.

Thanks
-Selim

On Wednesday, February 4, 2015 at 3:57:00 PM UTC-8, Selim Tuvi wrote:

Actually I fixed my query in Sense, and now I am getting the same
error:

GET /igo/task/_search
{
"query": {
"match_all": {}
},
"sort" : [
{"_script": {
"lang": "groovy",
"script": "doc['shot.org'].value",
"type": "string",
"order":"asc"
}}
]
}

Any ideas why? All documents should have the field "shot.org" for
this
doc type.

-Selim

--
You received this message because you are subscribed to the Google
Groups "elasticsearch"
group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/5f5d18dc-f4fc-4895-a037-95ddf6613fab%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/605995d9-d3ce-4b34-bdb7-f2932ec4a52c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.