Custom Scripts In Elastic Search

Elastic Stack Elasticsearch
1

Hi team,
I have used solr and I am evaluating elastic search to see if
it would fit our needs well. Solr lets us extendend valueSourceParser to
create function queries. I understand that elasticsearch lets us to plugin
scripts as described in this posthttp://stackoverflow.com/questions/17739240/does-elasticsearch-support-conditional-queries.
But the scripts in all the elastic search resources I have come across have
been pretty simple inline scripts like doc['field'] * 2.

I would like to invoke a function defined in a script for scoring every
document and I would like to pass few parameters to it. Consider the
following example.

genderScorer.py

Score['male'] = maleScore
Score['female'] = femaleScore

def score(docid):
gender = doc['gender'].value
return Score[gender]

My Query would use the script above for scoring like below

custom_score" : {
"query" : {
....
},
"params" : {
"maleScore" : 20,
"femaleScore" : 40
},
"script" : "genderScorer.py"
}

As mentioned earlier, I am not sure if defining a script like above is even
possible. So following are my questions

  1. Is above possible out of the box?
  2. If available, what are the different hooks/methods that I should
    implement for the scripts? Like sorting might require different methods
    while compared to an updater script or a scorer script.
  3. If not available, Is there any other way of accomplishing the same by
    pluging in the code. For instance, solr has a bunch of extendable pluginshttp://wiki.apache.org/solr/SolrPlugins#Classes_that_are_.27Pluggable.27
    .

Thanks,
Karthick

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

Hi Kathrick,

It is possible, though the semantics are somewhat different that what
describe. I assume you want to using python using the python scripting
plugin.

The script is run using an eval command
( http://www.jython.org/javadoc/org/python/util/PythonInterpreter.html#eval(java.lang.String)
) which is similar to python's eval (
2. Built-in Functions — Python 2.7.18 documentation ) . That means that
the value of the last statement is returned. The code is evaluated with
_score, doc etc. as variables in the context. I haven't tested it, but I
think this script should work:

Score= { "male": maleScore, "female": femaleScore }
Score[doc["gender"].value]

Hope this helps,
Boaz

On Friday, July 19, 2013 10:28:53 PM UTC+2, Karthick Duraisamy Soundararaj
wrote:

Hi team,
I have used solr and I am evaluating Elasticsearch to see if
it would fit our needs well. Solr lets us extendend valueSourceParser to
create function queries. I understand that elasticsearch lets us to plugin
scripts as described in this posthttp://stackoverflow.com/questions/17739240/does-elasticsearch-support-conditional-queries.
But the scripts in all the Elasticsearch resources I have come across have
been pretty simple inline scripts like doc['field'] * 2.

I would like to invoke a function defined in a script for scoring every
document and I would like to pass few parameters to it. Consider the
following example.

genderScorer.py

Score['male'] = maleScore
Score['female'] = femaleScore

def score(docid):
gender = doc['gender'].value
return Score[gender]

My Query would use the script above for scoring like below

custom_score" : {
"query" : {
....
},
"params" : {
"maleScore" : 20,
"femaleScore" : 40
},
"script" : "genderScorer.py"
}

As mentioned earlier, I am not sure if defining a script like above is
even possible. So following are my questions

  1. Is above possible out of the box?
  2. If available, what are the different hooks/methods that I should
    implement for the scripts? Like sorting might require different methods
    while compared to an updater script or a scorer script.
  3. If not available, Is there any other way of accomplishing the same by
    pluging in the code. For instance, solr has a bunch of extendable pluginshttp://wiki.apache.org/solr/SolrPlugins#Classes_that_are_.27Pluggable.27
    .

Thanks,
Karthick

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

Hi Karthick,

This is possible out of the box, though the semantics are slightly
different.I assume you are interested in doing this in python: ES uses a
java version of python's eval
( 2. Built-in Functions — Python 2.7.18 documentation ) . This means the
value of the last statement in the script is what used for scoring. ES
inject local variables for things like _script , doc etc.

I haven't tried it but this (or something very similar) should work:

Score = { 'male' : maleScore , ''female' : femaleScore }
Score[doc['gender'].value]

Cheers,
Boaz

On Friday, July 19, 2013 10:28:53 PM UTC+2, Karthick Duraisamy Soundararaj
wrote:

Hi team,
I have used solr and I am evaluating Elasticsearch to see if
it would fit our needs well. Solr lets us extendend valueSourceParser to
create function queries. I understand that elasticsearch lets us to plugin
scripts as described in this posthttp://stackoverflow.com/questions/17739240/does-elasticsearch-support-conditional-queries.
But the scripts in all the Elasticsearch resources I have come across have
been pretty simple inline scripts like doc['field'] * 2.

I would like to invoke a function defined in a script for scoring every
document and I would like to pass few parameters to it. Consider the
following example.

genderScorer.py

Score['male'] = maleScore
Score['female'] = femaleScore

def score(docid):
gender = doc['gender'].value
return Score[gender]

My Query would use the script above for scoring like below

custom_score" : {
"query" : {
....
},
"params" : {
"maleScore" : 20,
"femaleScore" : 40
},
"script" : "genderScorer.py"
}

As mentioned earlier, I am not sure if defining a script like above is
even possible. So following are my questions

  1. Is above possible out of the box?
  2. If available, what are the different hooks/methods that I should
    implement for the scripts? Like sorting might require different methods
    while compared to an updater script or a scorer script.
  3. If not available, Is there any other way of accomplishing the same by
    pluging in the code. For instance, solr has a bunch of extendable pluginshttp://wiki.apache.org/solr/SolrPlugins#Classes_that_are_.27Pluggable.27
    .

Thanks,
Karthick

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

4

Boaz,
Thanks for the reply. Good to know that an external script could
be used. But, lets say, I want to initialize the script when I start the
server and then invoke a particular function for queries, is it possible?

Thanks,
Karthick

On Mon, Jul 22, 2013 at 12:18 PM, Boaz Leskes b.leskes@gmail.com wrote:

Hi Karthick,

This is possible out of the box, though the semantics are slightly
different.I assume you are interested in doing this in python: ES uses a
java version of python's eval (
2. Built-in Functions — Python 2.7.18 documentation ) . This means the
value of the last statement in the script is what used for scoring. ES
inject local variables for things like _script , doc etc.

I haven't tried it but this (or something very similar) should work:

Score = { 'male' : maleScore , ''female' : femaleScore }
Score[doc['gender'].value]

Cheers,
Boaz

On Friday, July 19, 2013 10:28:53 PM UTC+2, Karthick Duraisamy Soundararaj
wrote:

Hi team,
I have used solr and I am evaluating Elasticsearch to see if
it would fit our needs well. Solr lets us extendend valueSourceParser to
create function queries. I understand that elasticsearch lets us to plugin
scripts as described in this posthttp://stackoverflow.com/questions/17739240/does-elasticsearch-support-conditional-queries.
But the scripts in all the Elasticsearch resources I have come across have
been pretty simple inline scripts like doc['field'] * 2.

I would like to invoke a function defined in a script for scoring every
document and I would like to pass few parameters to it. Consider the
following example.

genderScorer.py

Score['male'] = maleScore
Score['female'] = femaleScore

def score(docid):
gender = doc['gender'].value
return Score[gender]

My Query would use the script above for scoring like below

custom_score" : {
"query" : {

    ....
},
"params" : {

    "maleScore" : 20,

    "femaleScore" : 40

},
"script" : "genderScorer.py"

}

As mentioned earlier, I am not sure if defining a script like above is
even possible. So following are my questions

  1. Is above possible out of the box?
  2. If available, what are the different hooks/methods that I should
    implement for the scripts? Like sorting might require different methods
    while compared to an updater script or a scorer script.
  3. If not available, Is there any other way of accomplishing the same by
    pluging in the code. For instance, solr has a bunch of extendable pluginshttp://wiki.apache.org/solr/SolrPlugins#Classes_that_are_.27Pluggable.27
    .

Thanks,
Karthick

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/FJyV9QALpvs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

5

Hi Karthick,

After looking at it some more, it turns out I was a bit misleading. Sorry.

What I said is still true - ES uses jython eval, which is like cpython's
eval. However, it can only process single expressions. That means the
example I wrote will not run. You can still run the following:

{'male' : maleScore,'female':femaleScore}[doc['gender'].value]

but it has to be all in one line.

Also, I checked and the current implementation doesn't allow calling
functions which, due to the above, have to be defined in another file.

Cheers,
Boaz

On Mon, Jul 22, 2013 at 11:26 PM, Karthick Duraisamy Soundararaj <
karthick.soundararaj@gmail.com> wrote:

Boaz,
Thanks for the reply. Good to know that an external script could
be used. But, lets say, I want to initialize the script when I start the
server and then invoke a particular function for queries, is it possible?

Thanks,
Karthick

On Mon, Jul 22, 2013 at 12:18 PM, Boaz Leskes b.leskes@gmail.com wrote:

Hi Karthick,

This is possible out of the box, though the semantics are slightly
different.I assume you are interested in doing this in python: ES uses a
java version of python's eval (
2. Built-in Functions — Python 2.7.18 documentation ) . This means the
value of the last statement in the script is what used for scoring. ES
inject local variables for things like _script , doc etc.

I haven't tried it but this (or something very similar) should work:

Score = { 'male' : maleScore , ''female' : femaleScore }
Score[doc['gender'].value]

Cheers,
Boaz

On Friday, July 19, 2013 10:28:53 PM UTC+2, Karthick Duraisamy
Soundararaj wrote:

Hi team,
I have used solr and I am evaluating Elasticsearch to see
if it would fit our needs well. Solr lets us extendend valueSourceParser to
create function queries. I understand that elasticsearch lets us to plugin
scripts as described in this posthttp://stackoverflow.com/questions/17739240/does-elasticsearch-support-conditional-queries.
But the scripts in all the Elasticsearch resources I have come across have
been pretty simple inline scripts like doc['field'] * 2.

I would like to invoke a function defined in a script for scoring every
document and I would like to pass few parameters to it. Consider the
following example.

genderScorer.py

Score['male'] = maleScore
Score['female'] = femaleScore

def score(docid):
gender = doc['gender'].value
return Score[gender]

My Query would use the script above for scoring like below

custom_score" : {
"query" : {

    ....
},
"params" : {



    "maleScore" : 20,



    "femaleScore" : 40


},
"script" : "genderScorer.py"

}

As mentioned earlier, I am not sure if defining a script like above is
even possible. So following are my questions

  1. Is above possible out of the box?
  2. If available, what are the different hooks/methods that I should
    implement for the scripts? Like sorting might require different methods
    while compared to an updater script or a scorer script.
  3. If not available, Is there any other way of accomplishing the same by
    pluging in the code. For instance, solr has a bunch of extendable
    pluginshttp://wiki.apache.org/solr/SolrPlugins#Classes_that_are_.27Pluggable.27
    .

Thanks,
Karthick

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/FJyV9QALpvs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/FJyV9QALpvs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.