ES support for payloads

Hi,
I've tried to find the answer on this forum, but without much success.
Hope somebody will have an idea here.

I am trying to port some existing code in Solr, where particular tokens are
boosted in the string filed called "content".
The Solr based code on the indexing side uses term payload to store weight
information using DelimitedPayloadTokenFilter.
On the query side it implements* *org.apache.solr.search.QParserPlugin
and adds PayloadSimilarity so that in search, some documents have better
score if they have higher boost for matched terms.

What would be the best way to do this in ES?
I have indexing side done, that is fine, but I am struggling with Query
side.
How to tell ES to use PayloadSimilarity for "content" field, but not for
the other fields, if I search for term in all fields?
Do I need to register new query type / parser? How can I do that?

Thanks,
Milan

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

You can configure a custom similarity per field, see
http://www.elasticsearch.org/guide/reference/index-modules/similarity/
There is no need for new query type/parser.

How did you index? If you implement a custom similarity like
PayloadSimilarity, this is best done in a plugin.

Jörg

Am 13.07.13 12:55, schrieb Milan Agatonovic:

Hi,
I've tried to find the answer on this forum, but without much success.
Hope somebody will have an idea here.

I am trying to port some existing code in Solr, where particular
tokens are boosted in the string filed called "content".
The Solr based code on the indexing side uses term payload to store
weight information using /DelimitedPayloadTokenFilter./
/On the query side it
implements///org.apache.solr.search.QParserPlugin and adds
PayloadSimilarity so that in search, some documents have better score
if they have higher boost for matched terms.

What would be the best way to do this in ES?
I have indexing side done, that is fine, but I am struggling with
Query side.
How to tell ES to use PayloadSimilarity for "content" field, but not
for the other fields, if I search for term in all fields?
Do I need to register new query type / parser? How can I do that?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Thanks Jorg,
I am doing it in plugin.
I have custom TokenFilter which sets token payloads (TermBoostTokenFilter):
and custom similarity which uses payload for scoring (TermBoostSimilarity).
Text is analysed correctly

However after reading through the link you sent me I am not sure how to set
custom similarity per one field only.

Is my understanding correct?

  1. I need first to register my similarity type to be used on both index and
    search like this:

curl -XPOST 'http://host:port/tweeter/' -d '
{
"settings": {
"similarity": {
"index": {
"type":
"org.elasticsearch.index.similarity.TermBoostSimilarityProvider"
},
"search": {
"type":
"org.elasticsearch.index.similarity.TermBoostSimilarityProvider"
}
}
}
}

  1. then to specify my similarity which has
    type: org.elasticsearch.index.similarity.TermBoostSimilarityProvider

"similarity" : {
"my_similarity" : {
"type" :
"org.elasticsearch.index.similarity.TermBoostSimilarityProvider",
}
}

  1. and then specify that it is used only in my content field:

{
"book" : {
"properties" : {
"content" : { "type" : "string", "similarity" : "my_similarity" },
"title" : "type" : "string" }
}
}

and then I will have:

  1. my custom similarity used in index and search for only "content" field
  2. default (tf/idf) similarity used for both index and search for all other
    fields (in this case "title")

BTW, this post confuses me a bit since it suggests I need to implement a
query parser as well.
http://elasticsearch-users.115913.n3.nabble.com/custom-similarity-setting-does-not-work-with-version-0-20-2-td4029500.html

Thanks,
Milan

On Sat, Jul 13, 2013 at 2:25 PM, Jörg Prante joergprante@gmail.com wrote:

You can configure a custom similarity per field, see
http://www.elasticsearch.org/**guide/reference/index-modules/**similarity/http://www.elasticsearch.org/guide/reference/index-modules/similarity/
There is no need for new query type/parser.

How did you index? If you implement a custom similarity like
PayloadSimilarity, this is best done in a plugin.

Jörg

Am 13.07.13 12:55, schrieb Milan Agatonovic:

Hi,
I've tried to find the answer on this forum, but without much success.
Hope somebody will have an idea here.

I am trying to port some existing code in Solr, where particular tokens
are boosted in the string filed called "content".
The Solr based code on the indexing side uses term payload to store
weight information using /DelimitedPayloadTokenFilter./
/On the query side it implements///org.apache.solr.**search.QParserPlugin
and adds PayloadSimilarity so that in search, some documents have better
score if they have higher boost for matched terms.

What would be the best way to do this in ES?
I have indexing side done, that is fine, but I am struggling with Query
side.
How to tell ES to use PayloadSimilarity for "content" field, but not for
the other fields, if I search for term in all fields?
Do I need to register new query type / parser? How can I do that?

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@**googlegroups.comelasticsearch%2Bunsubscribe@googlegroups.com
.
For more options, visit https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out
.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.