# How to index \<term, weight\> pairs as \<term, payload\> pairs? And how to use the payload for scoring?

**URL:** https://discuss.elastic.co/t/how-to-index-term-weight-pairs-as-term-payload-pairs-and-how-to-use-the-payload-for-scoring/22561
**Category:** Elasticsearch
**Created:** [March 6, 2015, 4:32pm UTC](https://discuss.elastic.co/t/how-to-index-term-weight-pairs-as-term-payload-pairs-and-how-to-use-the-payload-for-scoring/22561 "2015-03-06T16:32:38Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![Devaraja\_Swami](https://avatars.discourse-cdn.com/v4/letter/d/b5a626/32.png) [@Devaraja\_Swami](https://discuss.elastic.co/u/Devaraja_Swami)
#### Post date: [March 6, 2015, 4:32pm UTC](https://discuss.elastic.co/t/how-to-index-term-weight-pairs-as-term-payload-pairs-and-how-to-use-the-payload-for-scoring/22561/1 "2015-03-06T16:32:38Z")

</div>

In my application, I have a map of \<term, weight\> pairs that I would like  
to index into the same field, say "f", where the term should be indexed "as  
is", ie, un-analyzed, and the weight should be indexed as the payload for  
that term. I only need payloads for that field, and not term vectors. Each  
document would have a different map of \<term, weight\> pairs to be indexed  
as term and payload into the field "F". I am using ES 1.4, with the Java  
API for both indexing and searching.  
I had been doing this (indexing and scoring) directly with Lucene thus far.  
However, after reading the ElasticSearch documentation, I could not find  
any API, or even an approach, for how to achieve payload indexing in  
ElasticSearch  
I did find some documentation on how to use scripting for extracting and  
using the payloads for scoring; this part would work, but I would prefer a  
Java API approach, because it would avoid the scripting performance hit.

Any help on this, especially the indexing part, would be appreciated.

-devarajaswami

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/e047c379-a23e-490f-a1e6-31d28dfa8c06%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/e047c379-a23e-490f-a1e6-31d28dfa8c06%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

### Author: ![Aurelien\_3](https://avatars.discourse-cdn.com/v4/letter/a/ce7236/32.png) [@Aurelien\_3](https://discuss.elastic.co/u/Aurelien_3)
#### Post date: [May 23, 2016, 3:44pm UTC](https://discuss.elastic.co/t/how-to-index-term-weight-pairs-as-term-payload-pairs-and-how-to-use-the-payload-for-scoring/22561/2 "2016-05-23T15:44:07Z")

</div>

Hi, I happened to have pretty much the same use case.

I used a home made plugin. One to index terms with payload. Actually you can use [https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-delimited-payload-tokenfilter.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-delimited-payload-tokenfilter.html) which does the same thing as mine.

Then I tried to use a custom similarity and overload :

```
@Override
public float scorePayload(int doc, int start, int end, BytesRef payload) {
  if (payload != null) {
return PayloadHelper.decodeFloat(payload.bytes, payload.offset);
} else {
  return 1.0F;
}
}

```

Unfortunately I never managed to make the payload taken into account in the scoring function.

I managed to use a script to score the payload. I added a script file in the server config scorePayload.groovy as follow :

```
score=0; 
for(term in search_terms) {
   termInfo = _index[field_name].get(term,_PAYLOADS | _CACHE); 
   for(pos in termInfo) { 
     score = score + pos.payloadAsFloat(1);
   }
 };
return log(score+2); 

```

And use the script as follow in my query :

```
  "query": {
    "function_score": {
      "score_mode": "multiply",
      "max_boost": 10,
      "functions": [
     {
      "script_score": {
          "script": {
                  "lang":"groovy",
                  "file": "scorePayloads",
                "params": { 
                      "field_name": "tags_payload",
                     "search_terms": ["term1"] }
          }
       }
      }
    ],
    "query": {
        "term": {
           "tags_payload": {
              "value": "term1"
           }
        }
     }
  }

```

I'm still wondering why the similarity foes not take into account my scorePayload.

---

<div class="post-metadata">

### Author: ![Aurelien\_3](https://avatars.discourse-cdn.com/v4/letter/a/ce7236/32.png) [@Aurelien\_3](https://discuss.elastic.co/u/Aurelien_3)
#### Post date: [May 23, 2016, 4:47pm UTC](https://discuss.elastic.co/t/how-to-index-term-weight-pairs-as-term-payload-pairs-and-how-to-use-the-payload-for-scoring/22561/3 "2016-05-23T16:47:19Z")

</div>

By the way, have you managed to handle a custom similarity into the scoring ?

---

<div class="post-metadata">

### Author: ![sdauletau](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sdauletau/32/24779_2.png) [@sdauletau](https://discuss.elastic.co/u/sdauletau)
#### Post date: [May 23, 2016, 6:16pm UTC](https://discuss.elastic.co/t/how-to-index-term-weight-pairs-as-term-payload-pairs-and-how-to-use-the-payload-for-scoring/22561/4 "2016-05-23T18:16:19Z")

</div>

I have an example of term position scoring.  
[https://github.com/sdauletau/elasticsearch-position-similarity](https://github.com/sdauletau/elasticsearch-position-similarity).

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 5, 2017, 10:49pm UTC](https://discuss.elastic.co/t/how-to-index-term-weight-pairs-as-term-payload-pairs-and-how-to-use-the-payload-for-scoring/22561/5 "2017-07-05T22:49:43Z")

</div>