Different search results from ES webservice and python's API pyres


(Abhishek Pratap) #1

Hi Guys

I am sort of confused why I am getting two different search results when I
query against same index using webservice and ES python's API pyres.

from webservice
curl -XGET localhost
:9201/ncbi_taxa_names/taxa_names/_search?q=tax_name:Myxococcushttp://128.55.54.143:9201/ncbi_taxa_names/taxa_names/_search?q=tax_name:Myxococcus
{"took":5,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":15,"
}}}} # only copying the #hits

from pyres:
conn.open_index("ncbi_taxa_names")
q = TermQuery("tax_name","Myxococcus")
results = conn.search(query=q)
for r in results:
print r
.....:
no hits :

Am I missing something ?

Thanks!
-Abhi

--


(David Pilato) #2

ES uses a QueryString (analyzed) but you create with Pyes a TermQuery ( not analyzed ).
Try this or use a TextQuery
TermQuery("tax_name","myxococcus")

David

Le 11 août 2012 à 02:04, Abhishek Pratap abhishek.vit@gmail.com a écrit :

Hi Guys

I am sort of confused why I am getting two different search results when I query against same index using webservice and ES python's API pyres.

from webservice
curl -XGET localhost:9201/ncbi_taxa_names/taxa_names/_search?q=tax_name:Myxococcus
{"took":5,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":15," }}}} # only copying the #hits

from pyres:
conn.open_index("ncbi_taxa_names")
q = TermQuery("tax_name","Myxococcus")
results = conn.search(query=q)
for r in results:
print r
.....:
no hits :

Am I missing something ?

Thanks!
-Abhi

--


(Abhishek Pratap) #3

Hi David

Thanks for your reply. Could you please elaborate a little bit more on what
you meant by analyzed string in the context of ES. I am new to ES and not
quite sure if I understand it fully.

Thanks!
-Abhi

On Friday, August 10, 2012 5:19:07 PM UTC-7, David Pilato wrote:

ES uses a QueryString (analyzed) but you create with Pyes a TermQuery (
not analyzed ).
Try this or use a TextQuery
TermQuery("tax_name","myxococcus")

David

Le 11 août 2012 à 02:04, Abhishek Pratap <abhish...@gmail.com<javascript:>>
a écrit :

Hi Guys

I am sort of confused why I am getting two different search results when I
query against same index using webservice and ES python's API pyres.

from webservice
curl -XGET localhost
:9201/ncbi_taxa_names/taxa_names/_search?q=tax_name:Myxococcushttp://128.55.54.143:9201/ncbi_taxa_names/taxa_names/_search?q=tax_name:Myxococcus
{"took":5,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":15,"
}}}} # only copying the #hits

from pyres:
conn.open_index("ncbi_taxa_names")
q = TermQuery("tax_name","Myxococcus")
results = conn.search(query=q)
for r in results:
print r
.....:
no hits :

Am I missing something ?

Thanks!
-Abhi

--

--


(David Pilato) #4

Analyze means to transform any text into tokens which will be indexed depending on your analysis rules.

Default ES analyzer will remove common english words (I, am, you, are, this...). It will also lowercase everything.

Then when searching, your text will be compared to indexed text.

So, you have in index : myxococcus .
If you search for Myxococcus without analyze, you won't find anything. If you analyze your search, Myxococcus will be transformed into myxococcus and it will match your doument.

hope this helps
David

Le 13 août 2012 à 21:38, Abhishek Pratap abhishek.vit@gmail.com a écrit :

Hi David

Thanks for your reply. Could you please elaborate a little bit more on what you meant by analyzed string in the context of ES. I am new to ES and not quite sure if I understand it fully.

Thanks!
-Abhi

On Friday, August 10, 2012 5:19:07 PM UTC-7, David Pilato wrote:
ES uses a QueryString (analyzed) but you create with Pyes a TermQuery ( not analyzed ).
Try this or use a TextQuery
TermQuery("tax_name","myxococcus")

David

Le 11 août 2012 à 02:04, Abhishek Pratap abhish...@gmail.com a écrit :

Hi Guys

I am sort of confused why I am getting two different search results when I query against same index using webservice and ES python's API pyres.

from webservice
curl -XGET localhost:9201/ncbi_taxa_names/taxa_names/_search?q=tax_name:Myxococcus
{"took":5,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":15," }}}} # only copying the #hits

from pyres:
conn.open_index("ncbi_taxa_names")
q = TermQuery("tax_name","Myxococcus")
results = conn.search(query=q)
for r in results:
print r
.....:
no hits :

Am I missing something ?

Thanks!
-Abhi

--

--


(Abhishek Pratap) #5

Thanks David. One more thing

Any doc on what ways I can query ES using pyes. The doc
on http://packages.python.org/pyes/manual/queries.html#quering has some
info but I dont see examples which I fund more useful.

For example how can I create an analyzed query string in pyes for search.

Thanks!
-Abhi

On Monday, August 13, 2012 1:32:47 PM UTC-7, David Pilato wrote:

Analyze means to transform any text into tokens which will be indexed
depending on your analysis rules.

Default ES analyzer will remove common english words (I, am, you, are,
this...). It will also lowercase everything.

Then when searching, your text will be compared to indexed text.

So, you have in index : myxococcus .
If you search for Myxococcus without analyze, you won't find anything. If
you analyze your search, Myxococcus will be transformed into myxococcus
and it will match your doument.

hope this helps
David

Le 13 août 2012 à 21:38, Abhishek Pratap <abhish...@gmail.com<javascript:>>
a écrit :

Hi David

Thanks for your reply. Could you please elaborate a little bit more on
what you meant by analyzed string in the context of ES. I am new to ES and
not quite sure if I understand it fully.

Thanks!
-Abhi

On Friday, August 10, 2012 5:19:07 PM UTC-7, David Pilato wrote:

ES uses a QueryString (analyzed) but you create with Pyes a TermQuery (
not analyzed ).
Try this or use a TextQuery
TermQuery("tax_name","myxococcus")

David

Le 11 août 2012 à 02:04, Abhishek Pratap abhish...@gmail.com a écrit :

Hi Guys

I am sort of confused why I am getting two different search results when
I query against same index using webservice and ES python's API pyres.

from webservice
curl -XGET localhost
:9201/ncbi_taxa_names/taxa_names/_search?q=tax_name:Myxococcushttp://128.55.54.143:9201/ncbi_taxa_names/taxa_names/_search?q=tax_name:Myxococcus
{"took":5,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":15,"
}}}} # only copying the #hits

from pyres:
conn.open_index("ncbi_taxa_names")
q = TermQuery("tax_name","Myxococcus")
results = conn.search(query=q)
for r in results:
print r
.....:
no hits :

Am I missing something ?

Thanks!
-Abhi

--

--

--


(David Pilato) #6

Sorry. I don't know pyes.
Anyone? Pyes gurus?

--
David

Le 13 août 2012 à 23:44, Abhishek Pratap abhishek.vit@gmail.com a écrit :

Thanks David. One more thing

Any doc on what ways I can query ES using pyes. The doc on http://packages.python.org/pyes/manual/queries.html#quering has some info but I dont see examples which I fund more useful.

For example how can I create an analyzed query string in pyes for search.

Thanks!
-Abhi

On Monday, August 13, 2012 1:32:47 PM UTC-7, David Pilato wrote:
Analyze means to transform any text into tokens which will be indexed depending on your analysis rules.

Default ES analyzer will remove common english words (I, am, you, are, this...). It will also lowercase everything.

Then when searching, your text will be compared to indexed text.

So, you have in index : myxococcus .
If you search for Myxococcus without analyze, you won't find anything. If you analyze your search, Myxococcus will be transformed into myxococcus and it will match your doument.

hope this helps
David

Le 13 août 2012 à 21:38, Abhishek Pratap abhish...@gmail.com a écrit :

Hi David

Thanks for your reply. Could you please elaborate a little bit more on what you meant by analyzed string in the context of ES. I am new to ES and not quite sure if I understand it fully.

Thanks!
-Abhi

On Friday, August 10, 2012 5:19:07 PM UTC-7, David Pilato wrote:
ES uses a QueryString (analyzed) but you create with Pyes a TermQuery ( not analyzed ).
Try this or use a TextQuery
TermQuery("tax_name","myxococcus")

David

Le 11 août 2012 à 02:04, Abhishek Pratap abhish...@gmail.com a écrit :

Hi Guys

I am sort of confused why I am getting two different search results when I query against same index using webservice and ES python's API pyres.

from webservice
curl -XGET localhost:9201/ncbi_taxa_names/taxa_names/_search?q=tax_name:Myxococcus
{"took":5,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":15," }}}} # only copying the #hits

from pyres:
conn.open_index("ncbi_taxa_names")
q = TermQuery("tax_name","Myxococcus")
results = conn.search(query=q)
for r in results:
print r
.....:
no hits :

Am I missing something ?

Thanks!
-Abhi

--

--

--


(system) #7