Python returns arrays after upgrading to ES 1.0


(G Kerekes) #1

We have just upgraded to 1.0 and when I run my previous queries in Python
(es.search()) I get back the dictionary values in squared brackets:

{u'_score': 1.0, u'_type': u'primary', u'_id': u'21792407', u'fields':
{u'outletstat': [u'Yes'], u'responseid': [u'21792407'], u'handling':
[u'Non-Handler'], u'response_modified': [u'2013-09-17 16:31:30'], u'lgai':
[u'Ikeja'], u'intid': [u'01133366']}, u'_index': u'log_data_all'}

Anyone experienced the same?

Thanks,
Geza

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/02835ab0-78ba-41f9-9ea0-f51918729a96%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Honza Král) #2

Hi Geza,

yes, this is correct behavior for elasticsearch now - since it wasn't
consistent before (sometimes you'd get back values, sometimes lists)
we changed it to be safe always (always return lists). This is not a
python-specific change, this is the difference in es output format.

If you're upgrading to 1.0 consider switching over to source filtering
instead of specifying fields, might look better to you:

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-source-filtering.html

Honza

On Thu, Feb 13, 2014 at 3:11 PM, G Kerekes kerekesg@gmail.com wrote:

We have just upgraded to 1.0 and when I run my previous queries in Python
(es.search()) I get back the dictionary values in squared brackets:

{u'_score': 1.0, u'_type': u'primary', u'_id': u'21792407', u'fields':
{u'outletstat': [u'Yes'], u'responseid': [u'21792407'], u'handling':
[u'Non-Handler'], u'response_modified': [u'2013-09-17 16:31:30'], u'lgai':
[u'Ikeja'], u'intid': [u'01133366']}, u'_index': u'log_data_all'}

Anyone experienced the same?

Thanks,
Geza

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/02835ab0-78ba-41f9-9ea0-f51918729a96%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CABfdDirKzUdW%2BqBOKPpYBPvG8jJPr7XkhJ52vah3usGnOhqAiw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(G Kerekes) #3

Hi Honza,

Thanks, make sense, it will be an easy change to make my code working.
Just to clarify, I would expect the squared brackets when I query from
browser or from Sense too, but the output look the same as in the previous
ES versions (no squared brackets):

"hits" : {
"total" : 84779,
"max_score" : 1.0,
"hits" : [ {
"_index" : "tns_log_data_all",
"_type" : "21792162",
"_id" : "primary",
"_score" : 1.0, "_source" : {"gps1": "6.4430759145232;3.3731560125946416;4.0;26.0;4.0;1", "cosmetic___drug": "", "file": "DestinationQC22 27-09-2013 10-49-21.xlsx.txt", "response_revision": "2", "nresponsename": "", "ontrade": "", "projectid": "1011286", "handling": "Non-Handler", "contactnum": "08038473333", "response_externalid": "", "response_received": "2013-09-25 16:40:05", "narea": "", "result": "Outllet could not be located/Not found/Not exist at all", "e__telecoms": "", "response_state": "COMPLETED", "lng": 3.373156, "outname": "", "nwlgai": "Apapa", "area": "wharf", "resultdum_outlet_closed_locked": "FALSE", "outletname": "", "project_revision": "3", "platform_version": "2.3.6", "responseid": "22112275", "surveycode": "", "int_flag": false, "lgai": "Apapa", "taskid": "410327", "other_category": "", "response_elapsed": "230", "handling2": "", "offtrade": "", "user_name": "wa0389", "channel": "", "resultdum_outlet_moved_demolished_reloca": "FALSE", "pcat_none1": "", "specialist": "", "club_institution": "", "user_externalid": "", "resultdum_outllet_could_not_be_located_n": "TRUE", "response_name": "", "intid": "01134717", "areachk": "Yes", "response_created": "2013-09-23 08:15:17", "outletstat": "No", "response_modified": "2013-09-25 16:39:00", "horeca": "", "address": "eleganza plaxa by aerofrome road", "pcat_beer": "", "response_mode": "LIVE", "platform_model": "GT-S5300", "pcat_spirit": "", "pcat_wine": "", "grocery": "", "app_version": "3.0.3", "lat": 6.443076, "lat_lng_accuracy": 4.0, "lgachk": "", "drinks_venues": "", "ncontactad": "", "comment1": "not existed", "respondentidd": "13145736", "pics": "Click to view", "platform_name": "android", "convenience": "", "pcat_carbonated_soft_drink": "", "response_comment": "", "responsename": "Mama chinasa", "pcat_malt": "", "resultdum_outlet_confirmed_but_contact_d": "FALSE", "resultdum_outlet_confirmed_with_all_info": "FALSE", "supid": "01120041", "ncontactphn": ""}
}

That's why I thought I was doing something wrong in Python.

Many thanks,
Geza

On Thursday, February 13, 2014 2:17:37 PM UTC, Honza Král wrote:

Hi Geza,

yes, this is correct behavior for elasticsearch now - since it wasn't
consistent before (sometimes you'd get back values, sometimes lists)
we changed it to be safe always (always return lists). This is not a
python-specific change, this is the difference in es output format.

If you're upgrading to 1.0 consider switching over to source filtering
instead of specifying fields, might look better to you:

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-source-filtering.html

Honza

On Thu, Feb 13, 2014 at 3:11 PM, G Kerekes <kere...@gmail.com<javascript:>>
wrote:

We have just upgraded to 1.0 and when I run my previous queries in
Python
(es.search()) I get back the dictionary values in squared brackets:

{u'_score': 1.0, u'_type': u'primary', u'_id': u'21792407', u'fields':
{u'outletstat': [u'Yes'], u'responseid': [u'21792407'], u'handling':
[u'Non-Handler'], u'response_modified': [u'2013-09-17 16:31:30'],
u'lgai':
[u'Ikeja'], u'intid': [u'01133366']}, u'_index': u'log_data_all'}

Anyone experienced the same?

Thanks,
Geza

--
You received this message because you are subscribed to the Google
Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit

https://groups.google.com/d/msgid/elasticsearch/02835ab0-78ba-41f9-9ea0-f51918729a96%40googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/18830359-53ae-4f28-9702-cda9e41016ca%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Honza Král) #4

Hi Geza,

if you request _source (default) you will get back what was in the
source when you indexed the document, the same as when you use source
filtering. If you, however, use fields to specify what you want
returned you will get back lists.

On Thu, Feb 13, 2014 at 3:38 PM, G Kerekes kerekesg@gmail.com wrote:

Hi Honza,

Thanks, make sense, it will be an easy change to make my code working.
Just to clarify, I would expect the squared brackets when I query from
browser or from Sense too, but the output look the same as in the previous
ES versions (no squared brackets):

"hits" : {
"total" : 84779,
"max_score" : 1.0,
"hits" : [ {
"_index" : "tns_log_data_all",
"_type" : "21792162",
"_id" : "primary",
"_score" : 1.0, "_source" : {"gps1":
"6.4430759145232;3.3731560125946416;4.0;26.0;4.0;1", "cosmetic___drug": "",
"file": "DestinationQC22 27-09-2013 10-49-21.xlsx.txt", "response_revision":
"2", "nresponsename": "", "ontrade": "", "projectid": "1011286", "handling":
"Non-Handler", "contactnum": "08038473333", "response_externalid": "",
"response_received": "2013-09-25 16:40:05", "narea": "", "result": "Outllet
could not be located/Not found/Not exist at all", "e__telecoms": "",
"response_state": "COMPLETED", "lng": 3.373156, "outname": "", "nwlgai":
"Apapa", "area": "wharf", "resultdum_outlet_closed_locked": "FALSE",
"outletname": "", "project_revision": "3", "platform_version": "2.3.6",
"responseid": "22112275", "surveycode": "", "int_flag": false, "lgai":
"Apapa", "taskid": "410327", "other_category": "", "response_elapsed":
"230", "handling2": "", "offtrade": "", "user_name": "wa0389", "channel":
"", "resultdum_outlet_moved_demolished_reloca": "FALSE", "pcat_none1": "",
"specialist": "", "club_institution": "", "user_externalid": "",
"resultdum_outllet_could_not_be_located_n": "TRUE", "response_name": "",
"intid": "01134717", "areachk": "Yes", "response_created": "2013-09-23
08:15:17", "outletstat": "No", "response_modified": "2013-09-25 16:39:00",
"horeca": "", "address": "eleganza plaxa by aerofrome road", "pcat_beer":
"", "response_mode": "LIVE", "platform_model": "GT-S5300", "pcat_spirit":
"", "pcat_wine": "", "grocery": "", "app_version": "3.0.3", "lat": 6.443076,
"lat_lng_accuracy": 4.0, "lgachk": "", "drinks_venues": "", "ncontactad":
"", "comment1": "not existed", "respondentidd": "13145736", "pics": "Click
to view", "platform_name": "android", "convenience": "",
"pcat_carbonated_soft_drink": "", "response_comment": "", "responsename":
"Mama chinasa", "pcat_malt": "", "resultdum_outlet_confirmed_but_contact_d":
"FALSE", "resultdum_outlet_confirmed_with_all_info": "FALSE", "supid":
"01120041", "ncontactphn": ""}
}

That's why I thought I was doing something wrong in Python.

Many thanks,
Geza

On Thursday, February 13, 2014 2:17:37 PM UTC, Honza Král wrote:

Hi Geza,

yes, this is correct behavior for elasticsearch now - since it wasn't
consistent before (sometimes you'd get back values, sometimes lists)
we changed it to be safe always (always return lists). This is not a
python-specific change, this is the difference in es output format.

If you're upgrading to 1.0 consider switching over to source filtering
instead of specifying fields, might look better to you:

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-source-filtering.html

Honza

On Thu, Feb 13, 2014 at 3:11 PM, G Kerekes kere...@gmail.com wrote:

We have just upgraded to 1.0 and when I run my previous queries in
Python
(es.search()) I get back the dictionary values in squared brackets:

{u'_score': 1.0, u'_type': u'primary', u'_id': u'21792407', u'fields':
{u'outletstat': [u'Yes'], u'responseid': [u'21792407'], u'handling':
[u'Non-Handler'], u'response_modified': [u'2013-09-17 16:31:30'],
u'lgai':
[u'Ikeja'], u'intid': [u'01133366']}, u'_index': u'log_data_all'}

Anyone experienced the same?

Thanks,
Geza

--
You received this message because you are subscribed to the Google
Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit

https://groups.google.com/d/msgid/elasticsearch/02835ab0-78ba-41f9-9ea0-f51918729a96%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/18830359-53ae-4f28-9702-cda9e41016ca%40googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CABfdDio9v%3DbednKQK_-gnLMChTStpdOd7cdSeKs4OxGkW7tq0Q%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(system) #5