Solr SearchComponent-like functionality?

Elastic Stack Elasticsearch
1

Hello,

A long time Solr user posted a good question about ES over on Sematext
Blog, about an equivalent of Solr's SearchComponents in ES:

I'm curious, too. Thanks.

Otis

Sematext is hiring Search Engineers -- http://sematext.com/about/jobs.html

2

Hi,

I am not Solr expert but to me it seems that SearchComponents in Solr are
about extensibility of out of the box functionality. If that is the case
then I would say that we can talk about plugins in ES world. Although there
is no official doc about how to implement custom plugins yet it is really
not difficult. Apart from that there are several plugins that are part of
distribution (river plugins, attachments mapper, ICU analysis, scripting
languages ... to name a few) and they can be used as an inspiration if a new
plugin implementation is needed.

My 2 cents.

Lukas

On Tue, Sep 6, 2011 at 5:35 PM, Otis Gospodnetic <otis.gospodnetic@gmail.com

wrote:

Hello,

A long time Solr user posted a good question about ES over on Sematext
Blog, about an equivalent of Solr's SearchComponents in ES:

Solr vs Elasticsearch: Performance Differences & More - Sematext

I'm curious, too. Thanks.

Otis

Sematext is hiring Search Engineers -- Jobs - Sematext

3

Hi Lukas,

Yes, SearchComponents are about extensibility, but specifically about
extending how queries are handled within Solr once Solr gets them. I
know ES has other types of plugins, and you've listed several of them,
but I'm wondering about which of them is SearchComponent-like.
I've looked at Elasticsearch Platform — Find real-time answers at scale | Elastic
, but couldn't find the answer to my Q there. Maybe I'm looking at
the wrong place?

Thanks,
Otis

Sematext is hiring Search Engineers -- Jobs - Sematext

On Sep 6, 2:57 pm, Lukáš Vlček lukas.vl...@gmail.com wrote:

Hi,

I am not Solr expert but to me it seems that SearchComponents in Solr are
about extensibility of out of the box functionality. If that is the case
then I would say that we can talk about plugins in ES world. Although there
is no official doc about how to implement custom plugins yet it is really
not difficult. Apart from that there are several plugins that are part of
distribution (river plugins, attachments mapper, ICU analysis, scripting
languages ... to name a few) and they can be used as an inspiration if a new
plugin implementation is needed.

My 2 cents.

Lukas

On Tue, Sep 6, 2011 at 5:35 PM, Otis Gospodnetic <otis.gospodne...@gmail.com

wrote:
Hello,

A long time Solr user posted a good question about ES over on Sematext
Blog, about an equivalent of Solr's SearchComponents in ES:

Solr vs Elasticsearch: Performance Differences & More - Sematext...

I'm curious, too. Thanks.

Otis

Sematext is hiring Search Engineers --Jobs - Sematext

4

Hi Otis,

So if I understand it correctly (providing my knowledge is quite limited
here) you are asking if

  1. it is possible to hook into query processing flow and inject or extend
    custom handlers for individual flow phases and
  2. if we can find in ES the same functionality which is currently provided
    by components listed here: SearchComponent - Solr - Apache Software Foundation (or
    here:
    http://lucene.apache.org/solr/api/org/apache/solr/handler/component/SearchComponent.html
    ).

As for #1, frankly, I do not know. I have been playing with plugins a bit
but did not have a chance to explore full potential of it yet. I remember
that Shay mentioned that not every aspect of ES is pluggable now but that is
all I know about it (personally, I did not hit the limits by myself yet, may
be I would if I wanted to employ Carrot2 clustering or something like that)

As for #2, if you are after one-to-one comparison of Solr SearchComponents
and ES then I think we would find some matches and also some misses. Still
it could be an interesting exercise to do (although we should be careful to
include only those features that do work well in distributed environment).
We could probably end up identifying new feature requests, so this can be
useful.

Regards,
Lukas

On Wed, Sep 7, 2011 at 6:17 PM, Otis Gospodnetic <otis.gospodnetic@gmail.com

wrote:

Hi Lukas,

Yes, SearchComponents are about extensibility, but specifically about
extending how queries are handled within Solr once Solr gets them. I
know ES has other types of plugins, and you've listed several of them,
but I'm wondering about which of them is SearchComponent-like.
I've looked at
Elasticsearch Platform — Find real-time answers at scale | Elastic
, but couldn't find the answer to my Q there. Maybe I'm looking at
the wrong place?

Thanks,
Otis

Sematext is hiring Search Engineers -- Jobs - Sematext

On Sep 6, 2:57 pm, Lukáš Vlček lukas.vl...@gmail.com wrote:

Hi,

I am not Solr expert but to me it seems that SearchComponents in Solr are
about extensibility of out of the box functionality. If that is the case
then I would say that we can talk about plugins in ES world. Although
there
is no official doc about how to implement custom plugins yet it is really
not difficult. Apart from that there are several plugins that are part of
distribution (river plugins, attachments mapper, ICU analysis, scripting
languages ... to name a few) and they can be used as an inspiration if a
new
plugin implementation is needed.

My 2 cents.

Lukas

On Tue, Sep 6, 2011 at 5:35 PM, Otis Gospodnetic <
otis.gospodne...@gmail.com

wrote:
Hello,

A long time Solr user posted a good question about ES over on Sematext
Blog, about an equivalent of Solr's SearchComponents in ES:

Solr vs Elasticsearch: Performance Differences & More - Sematext.
..

I'm curious, too. Thanks.

Otis

Sematext is hiring Search Engineers --
Jobs - Sematext

5

I would like to influence the ranking with few fields that are not stored
in the index (eg click data for keyword-documents). I have used custom
SearchComponent in Solr to implement similar functionality in the past. I
am wondering how can i achieve the same in Elasticsearch.

I know this thread is a very old thread, but i didnt find much information
on how to do custom scoring (in elasticsearch) with data thats not stored
in the index. This thread looked very relevant to my requirement, so trying
to see whether you guys have solved similar requirements with elasticsearch.

Thanks
Srini

On Wednesday, September 7, 2011 12:18:09 PM UTC-7, Lukáš Vlček wrote:

Hi Otis,

So if I understand it correctly (providing my knowledge is quite limited
here) you are asking if

  1. it is possible to hook into query processing flow and inject or extend
    custom handlers for individual flow phases and
  2. if we can find in ES the same functionality which is currently provided
    by components listed here: SearchComponent - Solr - Apache Software Foundation (or
    here:
    http://lucene.apache.org/solr/api/org/apache/solr/handler/component/SearchComponent.html
    ).

As for #1, frankly, I do not know. I have been playing with plugins a bit
but did not have a chance to explore full potential of it yet. I remember
that Shay mentioned that not every aspect of ES is pluggable now but that
is all I know about it (personally, I did not hit the limits by myself yet,
may be I would if I wanted to employ Carrot2 clustering or something like
that)

As for #2, if you are after one-to-one comparison of Solr SearchComponents
and ES then I think we would find some matches and also some misses. Still
it could be an interesting exercise to do (although we should be careful to
include only those features that do work well in distributed environment).
We could probably end up identifying new feature requests, so this can be
useful.

Regards,
Lukas

On Wed, Sep 7, 2011 at 6:17 PM, Otis Gospodnetic <otis.gos...@gmail.com<javascript:>

wrote:

Hi Lukas,

Yes, SearchComponents are about extensibility, but specifically about
extending how queries are handled within Solr once Solr gets them. I
know ES has other types of plugins, and you've listed several of them,
but I'm wondering about which of them is SearchComponent-like.
I've looked at
Elasticsearch Platform — Find real-time answers at scale | Elastic
, but couldn't find the answer to my Q there. Maybe I'm looking at
the wrong place?

Thanks,
Otis

Sematext is hiring Search Engineers --
Jobs - Sematext

On Sep 6, 2:57 pm, Lukáš Vlček lukas.vl...@gmail.com wrote:

Hi,

I am not Solr expert but to me it seems that SearchComponents in Solr
are
about extensibility of out of the box functionality. If that is the case
then I would say that we can talk about plugins in ES world. Although
there
is no official doc about how to implement custom plugins yet it is
really
not difficult. Apart from that there are several plugins that are part
of
distribution (river plugins, attachments mapper, ICU analysis, scripting
languages ... to name a few) and they can be used as an inspiration if
a new
plugin implementation is needed.

My 2 cents.

Lukas

On Tue, Sep 6, 2011 at 5:35 PM, Otis Gospodnetic <
otis.gospodne...@gmail.com

wrote:
Hello,

A long time Solr user posted a good question about ES over on Sematext
Blog, about an equivalent of Solr's SearchComponents in ES:

Solr vs Elasticsearch: Performance Differences & More - Sematext.
..

I'm curious, too. Thanks.

Otis

Sematext is hiring Search Engineers --
Jobs - Sematext

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/16add2bc-c629-4613-934f-004c8cc749df%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

6

Yes, you can use the Function Score Query [1] in combination with a native
script written in java [2]. With the native script you can basically do
whatever you want, but be careful you can significantly impact your query
performance if you are not careful.

[1]

[2]

Thanks,
Matt Weber

On Thu, Apr 17, 2014 at 11:54 PM, Srinivasan Ramaswamy
ursvasan@gmail.comwrote:

I would like to influence the ranking with few fields that are not stored
in the index (eg click data for keyword-documents). I have used custom
SearchComponent in Solr to implement similar functionality in the past. I
am wondering how can i achieve the same in Elasticsearch.

I know this thread is a very old thread, but i didnt find much information
on how to do custom scoring (in elasticsearch) with data thats not stored
in the index. This thread looked very relevant to my requirement, so trying
to see whether you guys have solved similar requirements with elasticsearch.

Thanks
Srini

On Wednesday, September 7, 2011 12:18:09 PM UTC-7, Lukáš Vlček wrote:

Hi Otis,

So if I understand it correctly (providing my knowledge is quite limited
here) you are asking if

  1. it is possible to hook into query processing flow and inject or extend
    custom handlers for individual flow phases and
  2. if we can find in ES the same functionality which is currently
    provided by components listed here: http://wiki.apache.org/
    solr/SearchComponent (or here: Welcome to Apache Solr - Apache Solr
    api/org/apache/solr/handler/component/SearchComponent.html).

As for #1, frankly, I do not know. I have been playing with plugins a bit
but did not have a chance to explore full potential of it yet. I remember
that Shay mentioned that not every aspect of ES is pluggable now but that
is all I know about it (personally, I did not hit the limits by myself yet,
may be I would if I wanted to employ Carrot2 clustering or something like
that)

As for #2, if you are after one-to-one comparison of Solr
SearchComponents and ES then I think we would find some matches and also
some misses. Still it could be an interesting exercise to do (although we
should be careful to include only those features that do work well in
distributed environment). We could probably end up identifying new feature
requests, so this can be useful.

Regards,
Lukas

On Wed, Sep 7, 2011 at 6:17 PM, Otis Gospodnetic otis.gos...@gmail.comwrote:

Hi Lukas,

Yes, SearchComponents are about extensibility, but specifically about
extending how queries are handled within Solr once Solr gets them. I
know ES has other types of plugins, and you've listed several of them,
but I'm wondering about which of them is SearchComponent-like.
I've looked at Elasticsearch Platform — Find real-time answers at scale | Elastic
plugins.html
, but couldn't find the answer to my Q there. Maybe I'm looking at
the wrong place?

Thanks,
Otis

Sematext is hiring Search Engineers -- About Sematext - Solr / Elasticsearch Experts
jobs.html

On Sep 6, 2:57 pm, Lukáš Vlček lukas.vl...@gmail.com wrote:

Hi,

I am not Solr expert but to me it seems that SearchComponents in Solr
are
about extensibility of out of the box functionality. If that is the
case
then I would say that we can talk about plugins in ES world. Although
there
is no official doc about how to implement custom plugins yet it is
really
not difficult. Apart from that there are several plugins that are part
of
distribution (river plugins, attachments mapper, ICU analysis,
scripting
languages ... to name a few) and they can be used as an inspiration if
a new
plugin implementation is needed.

My 2 cents.

Lukas

On Tue, Sep 6, 2011 at 5:35 PM, Otis Gospodnetic <
otis.gospodne...@gmail.com

wrote:
Hello,

A long time Solr user posted a good question about ES over on
Sematext
Blog, about an equivalent of Solr's SearchComponents in ES:

Why Elastic Data Is Not Spreading Between Nodes - Sematext
distributed-lucene...

I'm curious, too. Thanks.

Otis

Sematext is hiring Search Engineers --About Sematext - Solr / Elasticsearch Experts
jobs.html

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/16add2bc-c629-4613-934f-004c8cc749df%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/16add2bc-c629-4613-934f-004c8cc749df%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAJ3KEoAeaSQcJyF2KTnxqKKR5uEAJYkSLftouAwJNwM%3Dd-rVGQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

7

Thats great, thanks for your reply. This looks like a good solution for my
requirement ! Is this script applied in each shard ? I want to apply this
function to all the documents so that the Top N picked from each shard is
picked by my custom score.

Also, can you elaborate a little bit on "be careful you can significantly
impact your query performance if you are not careful". I would like to
understand the best practices there.

On Friday, April 18, 2014 8:14:54 AM UTC-7, Matt Weber wrote:

Yes, you can use the Function Score Query [1] in combination with a native
script written in java [2]. With the native script you can basically do
whatever you want, but be careful you can significantly impact your query
performance if you are not careful.

[1]
Elasticsearch Platform — Find real-time answers at scale | Elastic
[2]
Elasticsearch Platform — Find real-time answers at scale | Elastic

Thanks,
Matt Weber

On Thu, Apr 17, 2014 at 11:54 PM, Srinivasan Ramaswamy <ursv...@gmail.com<javascript:>

wrote:

I would like to influence the ranking with few fields that are not stored
in the index (eg click data for keyword-documents). I have used custom
SearchComponent in Solr to implement similar functionality in the past. I
am wondering how can i achieve the same in Elasticsearch.

I know this thread is a very old thread, but i didnt find much
information on how to do custom scoring (in elasticsearch) with data thats
not stored in the index. This thread looked very relevant to my
requirement, so trying to see whether you guys have solved similar
requirements with elasticsearch.

Thanks
Srini

On Wednesday, September 7, 2011 12:18:09 PM UTC-7, Lukáš Vlček wrote:

Hi Otis,

So if I understand it correctly (providing my knowledge is quite limited
here) you are asking if

  1. it is possible to hook into query processing flow and inject or
    extend custom handlers for individual flow phases and
  2. if we can find in ES the same functionality which is currently
    provided by components listed here: http://wiki.apache.org/
    solr/SearchComponent (or here: Welcome to Apache Solr - Apache Solr
    api/org/apache/solr/handler/component/SearchComponent.html).

As for #1, frankly, I do not know. I have been playing with plugins a
bit but did not have a chance to explore full potential of it yet. I
remember that Shay mentioned that not every aspect of ES is pluggable now
but that is all I know about it (personally, I did not hit the limits by
myself yet, may be I would if I wanted to employ Carrot2 clustering or
something like that)

As for #2, if you are after one-to-one comparison of Solr
SearchComponents and ES then I think we would find some matches and also
some misses. Still it could be an interesting exercise to do (although we
should be careful to include only those features that do work well in
distributed environment). We could probably end up identifying new feature
requests, so this can be useful.

Regards,
Lukas

On Wed, Sep 7, 2011 at 6:17 PM, Otis Gospodnetic otis.gos...@gmail.comwrote:

Hi Lukas,

Yes, SearchComponents are about extensibility, but specifically about
extending how queries are handled within Solr once Solr gets them. I
know ES has other types of plugins, and you've listed several of them,
but I'm wondering about which of them is SearchComponent-like.
I've looked at Elasticsearch Platform — Find real-time answers at scale | Elastic
plugins.html
, but couldn't find the answer to my Q there. Maybe I'm looking at
the wrong place?

Thanks,
Otis

Sematext is hiring Search Engineers -- About Sematext - Solr / Elasticsearch Experts
jobs.html

On Sep 6, 2:57 pm, Lukáš Vlček lukas.vl...@gmail.com wrote:

Hi,

I am not Solr expert but to me it seems that SearchComponents in Solr
are
about extensibility of out of the box functionality. If that is the
case
then I would say that we can talk about plugins in ES world. Although
there
is no official doc about how to implement custom plugins yet it is
really
not difficult. Apart from that there are several plugins that are
part of
distribution (river plugins, attachments mapper, ICU analysis,
scripting
languages ... to name a few) and they can be used as an inspiration
if a new
plugin implementation is needed.

My 2 cents.

Lukas

On Tue, Sep 6, 2011 at 5:35 PM, Otis Gospodnetic <
otis.gospodne...@gmail.com

wrote:
Hello,

A long time Solr user posted a good question about ES over on
Sematext
Blog, about an equivalent of Solr's SearchComponents in ES:

Why Elastic Data Is Not Spreading Between Nodes - Sematext
distributed-lucene...

I'm curious, too. Thanks.

Otis

Sematext is hiring Search Engineers --About Sematext - Solr / Elasticsearch Experts
jobs.html

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/16add2bc-c629-4613-934f-004c8cc749df%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/16add2bc-c629-4613-934f-004c8cc749df%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/c2f61e06-f419-4e7d-95ed-a92ca14261bc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

8

Well, the scripts runs against all matching documents of the query so you
can do a match_all query [1] to have the logic applied to all your
documents. This is going to be expensive though, so try to filter out as
many documents as possible before applying the custom scoring. Maybe even
perform a rescore [2] on the top X docs. It really all depends on your
requirements though. Run some tests and tune based on those results.

When I said to be careful. I mean don't do a lot of blocking IO or long
running calculations as the script is ran against each matching document.
Cache results and make the script return as quick as possible.

[1]

[2]

Thanks,
Matt Weber

On Fri, Apr 18, 2014 at 9:46 AM, Srinivasan Ramaswamy ursvasan@gmail.comwrote:

Thats great, thanks for your reply. This looks like a good solution for my
requirement ! Is this script applied in each shard ? I want to apply this
function to all the documents so that the Top N picked from each shard is
picked by my custom score.

Also, can you elaborate a little bit on "be careful you can significantly
impact your query performance if you are not careful". I would like to
understand the best practices there.

On Friday, April 18, 2014 8:14:54 AM UTC-7, Matt Weber wrote:

Yes, you can use the Function Score Query [1] in combination with a
native script written in java [2]. With the native script you can
basically do whatever you want, but be careful you can significantly impact
your query performance if you are not careful.

[1] Elasticsearch Platform — Find real-time answers at scale | Elastic
reference/current/query-dsl-function-score-query.html
[2] Elasticsearch Platform — Find real-time answers at scale | Elastic
reference/current/modules-scripting.html#_native_java_scripts

Thanks,
Matt Weber

On Thu, Apr 17, 2014 at 11:54 PM, Srinivasan Ramaswamy <ursv...@gmail.com

wrote:

I would like to influence the ranking with few fields that are not
stored in the index (eg click data for keyword-documents). I have used
custom SearchComponent in Solr to implement similar functionality in the
past. I am wondering how can i achieve the same in Elasticsearch.

I know this thread is a very old thread, but i didnt find much
information on how to do custom scoring (in elasticsearch) with data thats
not stored in the index. This thread looked very relevant to my
requirement, so trying to see whether you guys have solved similar
requirements with elasticsearch.

Thanks
Srini

On Wednesday, September 7, 2011 12:18:09 PM UTC-7, Lukáš Vlček wrote:

Hi Otis,

So if I understand it correctly (providing my knowledge is quite
limited here) you are asking if

  1. it is possible to hook into query processing flow and inject or
    extend custom handlers for individual flow phases and
  2. if we can find in ES the same functionality which is currently
    provided by components listed here: http://wiki.apache.org/s
    olr/SearchComponent (or here: Welcome to Apache Solr - Apache Solr
    api/org/apache/solr/handler/component/SearchComponent.html).

As for #1, frankly, I do not know. I have been playing with plugins a
bit but did not have a chance to explore full potential of it yet. I
remember that Shay mentioned that not every aspect of ES is pluggable now
but that is all I know about it (personally, I did not hit the limits by
myself yet, may be I would if I wanted to employ Carrot2 clustering or
something like that)

As for #2, if you are after one-to-one comparison of Solr
SearchComponents and ES then I think we would find some matches and also
some misses. Still it could be an interesting exercise to do (although we
should be careful to include only those features that do work well in
distributed environment). We could probably end up identifying new feature
requests, so this can be useful.

Regards,
Lukas

On Wed, Sep 7, 2011 at 6:17 PM, Otis Gospodnetic <otis.gos...@gmail.com

wrote:

Hi Lukas,

Yes, SearchComponents are about extensibility, but specifically about
extending how queries are handled within Solr once Solr gets them. I
know ES has other types of plugins, and you've listed several of them,
but I'm wondering about which of them is SearchComponent-like.
I've looked at Elasticsearch Platform — Find real-time answers at scale | Elastic
plugins.html
, but couldn't find the answer to my Q there. Maybe I'm looking at
the wrong place?

Thanks,
Otis

Sematext is hiring Search Engineers -- http://sematext.com/about/jobs
.html

On Sep 6, 2:57 pm, Lukáš Vlček lukas.vl...@gmail.com wrote:

Hi,

I am not Solr expert but to me it seems that SearchComponents in
Solr are
about extensibility of out of the box functionality. If that is the
case
then I would say that we can talk about plugins in ES world.
Although there
is no official doc about how to implement custom plugins yet it is
really
not difficult. Apart from that there are several plugins that are
part of
distribution (river plugins, attachments mapper, ICU analysis,
scripting
languages ... to name a few) and they can be used as an inspiration
if a new
plugin implementation is needed.

My 2 cents.

Lukas

On Tue, Sep 6, 2011 at 5:35 PM, Otis Gospodnetic <
otis.gospodne...@gmail.com

wrote:
Hello,

A long time Solr user posted a good question about ES over on
Sematext
Blog, about an equivalent of Solr's SearchComponents in ES:

http://blog.sematext.com/2010/05/03/elastic-search-distribu
ted-lucene...

I'm curious, too. Thanks.

Otis

Sematext is hiring Search Engineers --http://sematext.com/about/jo
bs.html

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/16add2bc-c629-4613-934f-004c8cc749df%
40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/16add2bc-c629-4613-934f-004c8cc749df%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/c2f61e06-f419-4e7d-95ed-a92ca14261bc%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/c2f61e06-f419-4e7d-95ed-a92ca14261bc%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAJ3KEoCxWq7SiLScmv%3DS%2BBeyBjpNXoinAbNVW4MvaBb063i-nQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

9

Thanks for your answer ! I have some followup questions

  1. I wrote a custom script to return a constant score of 2.5. It works for
    the following query

{
"query" : {
"function_score": {
"query":{
"match_all":{}
},
"script_score": {
"lang":"native",
"script": "constant-score"
}
}
}
}

in the result i see the value of _score is "2.5" for all docs. But when i
include a query string it doesnt work

{
"query" : {
"function_score": {
"query":{
"query_string":{
"query": "dog"
}
},
"script_score": {
"lang":"native",
"script": "constant-score"
}
}
}
}

  1. I would like to access the search keyword (if possible analyzed version)
    to some custom processing depending on the keyword. is that possible ?

  2. You mentioned I can consider rescore api, I am interested in trying that
    out. I have some external data stored outside the index (in-memory) , how
    do i make the rescore function pick up data from that external data
    structure instead of index ?

Thanks in advance for any help !

Thanks
Srini

On Friday, April 18, 2014 11:48:25 AM UTC-7, Matt Weber wrote:

Well, the scripts runs against all matching documents of the query so you
can do a match_all query [1] to have the logic applied to all your
documents. This is going to be expensive though, so try to filter out as
many documents as possible before applying the custom scoring. Maybe even
perform a rescore [2] on the top X docs. It really all depends on your
requirements though. Run some tests and tune based on those results.

When I said to be careful. I mean don't do a lot of blocking IO or long
running calculations as the script is ran against each matching document.
Cache results and make the script return as quick as possible.

[1]
Elasticsearch Platform — Find real-time answers at scale | Elastic
[2]
Elasticsearch Platform — Find real-time answers at scale | Elastic

Thanks,
Matt Weber

On Fri, Apr 18, 2014 at 9:46 AM, Srinivasan Ramaswamy <ursv...@gmail.com<javascript:>

wrote:

Thats great, thanks for your reply. This looks like a good solution for
my requirement ! Is this script applied in each shard ? I want to apply
this function to all the documents so that the Top N picked from each shard
is picked by my custom score.

Also, can you elaborate a little bit on "be careful you can significantly
impact your query performance if you are not careful". I would like to
understand the best practices there.

On Friday, April 18, 2014 8:14:54 AM UTC-7, Matt Weber wrote:

Yes, you can use the Function Score Query [1] in combination with a
native script written in java [2]. With the native script you can
basically do whatever you want, but be careful you can significantly impact
your query performance if you are not careful.

[1] Elasticsearch Platform — Find real-time answers at scale | Elastic
reference/current/query-dsl-function-score-query.html
[2] Elasticsearch Platform — Find real-time answers at scale | Elastic
reference/current/modules-scripting.html#_native_java_scripts

Thanks,
Matt Weber

On Thu, Apr 17, 2014 at 11:54 PM, Srinivasan Ramaswamy <
ursv...@gmail.com> wrote:

I would like to influence the ranking with few fields that are not
stored in the index (eg click data for keyword-documents). I have used
custom SearchComponent in Solr to implement similar functionality in the
past. I am wondering how can i achieve the same in Elasticsearch.

I know this thread is a very old thread, but i didnt find much
information on how to do custom scoring (in elasticsearch) with data thats
not stored in the index. This thread looked very relevant to my
requirement, so trying to see whether you guys have solved similar
requirements with elasticsearch.

Thanks
Srini

On Wednesday, September 7, 2011 12:18:09 PM UTC-7, Lukáš Vlček wrote:

Hi Otis,

So if I understand it correctly (providing my knowledge is quite
limited here) you are asking if

  1. it is possible to hook into query processing flow and inject or
    extend custom handlers for individual flow phases and
  2. if we can find in ES the same functionality which is currently
    provided by components listed here: http://wiki.apache.org/s
    olr/SearchComponent (or here: Welcome to Apache Solr - Apache Solr
    api/org/apache/solr/handler/component/SearchComponent.html).

As for #1, frankly, I do not know. I have been playing with plugins a
bit but did not have a chance to explore full potential of it yet. I
remember that Shay mentioned that not every aspect of ES is pluggable now
but that is all I know about it (personally, I did not hit the limits by
myself yet, may be I would if I wanted to employ Carrot2 clustering or
something like that)

As for #2, if you are after one-to-one comparison of Solr
SearchComponents and ES then I think we would find some matches and also
some misses. Still it could be an interesting exercise to do (although we
should be careful to include only those features that do work well in
distributed environment). We could probably end up identifying new feature
requests, so this can be useful.

Regards,
Lukas

On Wed, Sep 7, 2011 at 6:17 PM, Otis Gospodnetic <
otis.gos...@gmail.com> wrote:

Hi Lukas,

Yes, SearchComponents are about extensibility, but specifically about
extending how queries are handled within Solr once Solr gets them. I
know ES has other types of plugins, and you've listed several of them,
but I'm wondering about which of them is SearchComponent-like.
I've looked at Elasticsearch Platform — Find real-time answers at scale | Elastic
plugins.html
, but couldn't find the answer to my Q there. Maybe I'm looking at
the wrong place?

Thanks,
Otis

Sematext is hiring Search Engineers -- http://sematext.com/about/jobs
.html

On Sep 6, 2:57 pm, Lukáš Vlček lukas.vl...@gmail.com wrote:

Hi,

I am not Solr expert but to me it seems that SearchComponents in
Solr are
about extensibility of out of the box functionality. If that is the
case
then I would say that we can talk about plugins in ES world.
Although there
is no official doc about how to implement custom plugins yet it is
really
not difficult. Apart from that there are several plugins that are
part of
distribution (river plugins, attachments mapper, ICU analysis,
scripting
languages ... to name a few) and they can be used as an inspiration
if a new
plugin implementation is needed.

My 2 cents.

Lukas

On Tue, Sep 6, 2011 at 5:35 PM, Otis Gospodnetic <
otis.gospodne...@gmail.com

wrote:
Hello,

A long time Solr user posted a good question about ES over on
Sematext
Blog, about an equivalent of Solr's SearchComponents in ES:

http://blog.sematext.com/2010/05/03/elastic-search-distribu
ted-lucene...

I'm curious, too. Thanks.

Otis

Sematext is hiring Search Engineers --About Sematext - Solr / Elasticsearch Experts
jobs.html

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/16add2bc-c629-4613-934f-004c8cc749df%
40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/16add2bc-c629-4613-934f-004c8cc749df%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/c2f61e06-f419-4e7d-95ed-a92ca14261bc%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/c2f61e06-f419-4e7d-95ed-a92ca14261bc%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5287e96f-7536-4ec8-b2ad-debca06c7af9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.