Custom Score

Elastic Stack Elasticsearch
1

I'm using Elastic Search to index user documents. We're looking to
implement a custom score based on the closeness of other users. So we'll
compare one user to another. Each user has the following fields:

interests: ["Lucene", "ElasticSearch"...]

Python Script
len(set(interests).intersection(set(doc['interests'].value)))

The script is attempting to find the common interests between the two
people, however it keeps giving an error. I'm using the Ruby Tire gem,
which produces the following command:

[REQUEST FAILED] curl -X GET
"http://localhost:9200/users/user/_search?pretty=true" -d
'{"query":{"custom_score":{"params":{"interests":["Entrepreneurship","Travel","Technology","Startups"]},"script":"meeteor","lang":"python","query":{"bool":{"should":[{"query_string":{"query":"jobs.employer:Facebook","default_operator":"AND"}},{"query_string":{"query":"jobs.employer:AOL","default_operator":"AND"}},{"query_string":{"query":"experienced_industries:Computer
Software","default_operator":"AND"}},{"query_string":{"query":"experienced_industries:Internet","default_operator":"AND"}}]}}}},"filter":{"and":[{"term":{"is_active":1}},{"term":{"is_email_confirmed":1}}]}}'
Tire::Search::SearchRequestFailed: 500 :
{"error":"SearchPhaseExecutionException[Failed to execute phase [query],
total failure; shardFailures {[7WFRorBIQoiuD2kl_RXGfQ][users][1]:
QueryPhaseExecutionException[[users][1]: query[filtered(custom score
(jobs.employer:facebook jobs.employer:aol (+experienced_industries:computer
+_all:software) experienced_industries:internet,function=script[meeteor],
params [{interests=[Entrepreneurship, Travel, Technology,
Startups]}]))->cache(_type:user)],from[0],size[10]: Query Failed [Failed to
execute main query]]; nested: }{[7WFRorBIQoiuD2kl_RXGfQ][users][2]:
QueryPhaseExecutionException[[users][2]: query[filtered(custom score
(jobs.employer:facebook jobs.employer:aol (+experienced_industries:computer
+_all:software) experienced_industries:internet,function=script[meeteor],
params [{interests=[Entrepreneurship, Travel, Technology,
Startups]}]))->cache(_type:user)],from[0],size[10]: Query Failed [Failed to
execute main query]]; nested: }]","status":500}

Any idea what's wrong?

2

Here's a formatted view of the json:

{
"query":{
"custom_score":{
"params":{
"interests":[
"Entrepreneurship",
"Travel",
"Technology",
"Startups"
]
},
"script":"meeteor",
"lang":"python",
"query":{
"bool":{
"should":[
{
"query_string":{
"query":"jobs.employer:Facebook",
"default_operator":"AND"
}
},
{
"query_string":{
"query":"jobs.employer:AOL",
"default_operator":"AND"
}
},
{
"query_string":{
"query":"experienced_industries:Computer Software",
"default_operator":"AND"
}
},
{
"query_string":{
"query":"experienced_industries:Internet",
"default_operator":"AND"
}
}
]
}
}
}
},
"filter":{
"and":[
{
"term":{
"is_active":1
}
},
{
"term":{
"is_email_confirmed":1
}
}
]
}
}

On Monday, April 30, 2012 10:29:29 PM UTC-4, Brandon Hilkert wrote:

I'm using Elastic Search to index user documents. We're looking to
implement a custom score based on the closeness of other users. So we'll
compare one user to another. Each user has the following fields:

interests: ["Lucene", "Elasticsearch"...]

Python Script
len(set(interests).intersection(set(doc['interests'].value)))

The script is attempting to find the common interests between the two
people, however it keeps giving an error. I'm using the Ruby Tire gem,
which produces the following command:

[REQUEST FAILED] curl -X GET "
http://localhost:9200/users/user/_search?pretty=true" -d
'{"query":{"custom_score":{"params":{"interests":["Entrepreneurship","Travel","Technology","Startups"]},"script":"meeteor","lang":"python","query":{"bool":{"should":[{"query_string":{"query":"jobs.employer:Facebook","default_operator":"AND"}},{"query_string":{"query":"jobs.employer:AOL","default_operator":"AND"}},{"query_string":{"query":"experienced_industries:Computer
Software","default_operator":"AND"}},{"query_string":{"query":"experienced_industries:Internet","default_operator":"AND"}}]}}}},"filter":{"and":[{"term":{"is_active":1}},{"term":{"is_email_confirmed":1}}]}}'
Tire::Search::SearchRequestFailed: 500 :
{"error":"SearchPhaseExecutionException[Failed to execute phase [query],
total failure; shardFailures {[7WFRorBIQoiuD2kl_RXGfQ][users][1]:
QueryPhaseExecutionException[[users][1]: query[filtered(custom score
(jobs.employer:facebook jobs.employer:aol (+experienced_industries:computer
+_all:software) experienced_industries:internet,function=script[meeteor],
params [{interests=[Entrepreneurship, Travel, Technology,
Startups]}]))->cache(_type:user)],from[0],size[10]: Query Failed [Failed to
execute main query]]; nested: }{[7WFRorBIQoiuD2kl_RXGfQ][users][2]:
QueryPhaseExecutionException[[users][2]: query[filtered(custom score
(jobs.employer:facebook jobs.employer:aol (+experienced_industries:computer
+_all:software) experienced_industries:internet,function=script[meeteor],
params [{interests=[Entrepreneurship, Travel, Technology,
Startups]}]))->cache(_type:user)],from[0],size[10]: Query Failed [Failed to
execute main query]]; nested: }]","status":500}

Any idea what's wrong?

3

On the server node, you should see a the full failure in the logs, can you
gist it?

On Tue, May 1, 2012 at 5:32 AM, Brandon Hilkert brandon@meeteor.com wrote:

Here's a formatted view of the json:

{
"query":{
"custom_score":{
"params":{
"interests":[
"Entrepreneurship",
"Travel",
"Technology",
"Startups"
]
},
"script":"meeteor",
"lang":"python",
"query":{
"bool":{
"should":[
{
"query_string":{
"query":"jobs.employer:Facebook",
"default_operator":"AND"
}
},
{
"query_string":{
"query":"jobs.employer:AOL",
"default_operator":"AND"
}
},
{
"query_string":{
"query":"experienced_industries:Computer Software"
,
"default_operator":"AND"
}
},
{
"query_string":{
"query":"experienced_industries:Internet",
"default_operator":"AND"
}
}
]
}
}
}
},
"filter":{
"and":[
{
"term":{
"is_active":1
}
},
{
"term":{
"is_email_confirmed":1
}
}
]
}
}

On Monday, April 30, 2012 10:29:29 PM UTC-4, Brandon Hilkert wrote:

I'm using Elastic Search to index user documents. We're looking to
implement a custom score based on the closeness of other users. So we'll
compare one user to another. Each user has the following fields:

interests: ["Lucene", "Elasticsearch"...]

Python Script
len(set(interests).**intersection(set(doc['**interests'].value)))

The script is attempting to find the common interests between the two
people, however it keeps giving an error. I'm using the Ruby Tire gem,
which produces the following command:

[REQUEST FAILED] curl -X GET "http://localhost:9200/users/**
user/search?pretty=truehttp://localhost:9200/users/user/_search?pretty=true"
-d '{"query":{"custom_score":{"params":{"interests":["
Entrepreneurship","Travel","Technology","Startups"]},"
script":"meeteor","lang":"python","query":{"bool":{"
should":[{"query_string":{"query":"jobs.employer:
Facebook","default_operator":"AND"}},{"query_string":{"
query":"jobs.employer:AOL","default_operator":"AND"}},{"
query_string":{"query":"**experienced_industries:**Computer
Software","default_operator":"AND"}},{"query_string":{"
query":"experiencedindustries:Internet","default_
operator":"AND"}}]}}}},"filter":{"and":[{"term":{"is_
active":1}},{"term":{"is_**email_confirmed":1}}]}}'
Tire::Search::SearchRequestFailed: 500 : {"error":"
SearchPhaseExecutionException[**Failed to execute phase [query], total
failure; shardFailures {[7WFRorBIQoiuD2kl_RXGfQ][**users][1]:
QueryPhaseExecutionException[[**users][1]: query[filtered(custom score
(jobs.employer:facebook jobs.employer:aol (+experienced_industries:**computer
+_all:software) experienced_industries:**internet,function=script[**meeteor],
params [{interests=[Entrepreneurship, Travel, Technology,
Startups]}]))->cache(_type:**user)],from[0],size[10]: Query Failed
[Failed to execute main query]]; nested: }{[7WFRorBIQoiuD2kl_RXGfQ][**users][2]:
QueryPhaseExecutionException[[**users][2]: query[filtered(custom score
(jobs.employer:facebook jobs.employer:aol (+experienced_industries:**computer
+_all:software) experienced_industries:**internet,function=script[**meeteor],
params [{interests=[Entrepreneurship, Travel, Technology,
Startups]}]))->cache(_type:**user)],from[0],size[10]: Query Failed
[Failed to execute main query]]; nested: }]","status":500}

Any idea what's wrong?

4

Hey Shay,

I ended up sorting it out. It was due to me not using the "values" method
when trying to compare an array field.

Thanks,
Brandon

On Wednesday, May 2, 2012 12:19:21 PM UTC-4, kimchy wrote:

On the server node, you should see a the full failure in the logs, can you
gist it?

On Tue, May 1, 2012 at 5:32 AM, Brandon Hilkert brandon@meeteor.comwrote:

Here's a formatted view of the json:

{
"query":{
"custom_score":{
"params":{
"interests":[
"Entrepreneurship",
"Travel",
"Technology",
"Startups"
]
},
"script":"meeteor",
"lang":"python",
"query":{
"bool":{
"should":[
{
"query_string":{
"query":"jobs.employer:Facebook",
"default_operator":"AND"
}
},
{
"query_string":{
"query":"jobs.employer:AOL",
"default_operator":"AND"
}
},
{
"query_string":{
"query":
"experienced_industries:Computer Software",
"default_operator":"AND"
}
},
{
"query_string":{
"query":"experienced_industries:Internet",
"default_operator":"AND"
}
}
]
}
}
}
},
"filter":{
"and":[
{
"term":{
"is_active":1
}
},
{
"term":{
"is_email_confirmed":1
}
}
]
}
}

On Monday, April 30, 2012 10:29:29 PM UTC-4, Brandon Hilkert wrote:

I'm using Elastic Search to index user documents. We're looking to
implement a custom score based on the closeness of other users. So we'll
compare one user to another. Each user has the following fields:

interests: ["Lucene", "Elasticsearch"...]

Python Script
len(set(interests).**intersection(set(doc['**interests'].value)))

The script is attempting to find the common interests between the two
people, however it keeps giving an error. I'm using the Ruby Tire gem,
which produces the following command:

[REQUEST FAILED] curl -X GET "http://localhost:9200/users/**
user/search?pretty=truehttp://localhost:9200/users/user/_search?pretty=true"
-d '{"query":{"custom_score":{"params":{"interests":["
Entrepreneurship","Travel","Technology","Startups"]},"
script":"meeteor","lang":"python","query":{"bool":{"
should":[{"query_string":{"query":"jobs.employer:
Facebook","default_operator":"AND"}},{"query_string":{"
query":"jobs.employer:AOL","default_operator":"AND"}},{"
query_string":{"query":"**experienced_industries:**Computer
Software","default_operator":"AND"}},{"query_string":{"
query":"experiencedindustries:Internet","default_
operator":"AND"}}]}}}},"filter":{"and":[{"term":{"is_
active":1}},{"term":{"is_**email_confirmed":1}}]}}'
Tire::Search::SearchRequestFailed: 500 : {"error":"
SearchPhaseExecutionException[**Failed to execute phase [query], total
failure; shardFailures {[7WFRorBIQoiuD2kl_RXGfQ][**users][1]:
QueryPhaseExecutionException[[**users][1]: query[filtered(custom score
(jobs.employer:facebook jobs.employer:aol (+experienced_industries:**computer
+_all:software) experienced_industries:**internet,function=script[**meeteor],
params [{interests=[Entrepreneurship, Travel, Technology,
Startups]}]))->cache(_type:**user)],from[0],size[10]: Query Failed
[Failed to execute main query]]; nested: }{[7WFRorBIQoiuD2kl_RXGfQ][**users][2]:
QueryPhaseExecutionException[[**users][2]: query[filtered(custom score
(jobs.employer:facebook jobs.employer:aol (+experienced_industries:**computer
+_all:software) experienced_industries:**internet,function=script[**meeteor],
params [{interests=[Entrepreneurship, Travel, Technology,
Startups]}]))->cache(_type:**user)],from[0],size[10]: Query Failed
[Failed to execute main query]]; nested: }]","status":500}

Any idea what's wrong?