Update settings working in 2.3.4 but not in 5.1

[Jimmy_Oentung]

Hi,
I'm trying to tune BM25 b and k1 values. the following works in 2.3.4:
{
"settings": {
"similarity": {
"my_bm25": {
"type": "BM25",
"b": b,
"k1": k
}
}
},
"mappings": {
docType: {
"properties": {
"title": {
"type": "string",
"similarity": "my_bm25"
},
"body": {
"type": "string",
"similarity": "my_bm25"
}
}
}
}
}

but when I ran it on elasticsearch 5.1 (i.e.: I already change the type "string" to "text") : I got the following error:
elasticsearch.exceptions.RequestError: TransportError(400, u'illegal_argument_exception', u'unknown setting [index.mappings.doc.properties.body.similarity] please check that any required plugins are installed, or check the breaking changes documentation for removed settings')

Anyone have insight on it?

[cbuescher (Christoph)]

Hi,

I just tried this on an empty 5.1 installation, works for me. Do you see any differences to what you did, except that I set some values for b, k1?

DELETE _all

PUT /index
{
  "settings": {
    "similarity": {
      "my_bm25": {
        "type": "BM25",
        "b": 0.76,
        "k1": 1.2
      }
    }
  },
  "mappings": {
    "docType": {
      "properties": {
        "title": {
          "type": "text",
          "similarity": "my_bm25"
        },
        "body": {
          "type": "text",
          "similarity": "my_bm25"
        }
      }
    }
  }
}

[Jimmy_Oentung]

Hi Christoph,
Thanks for the respond. Like you said, It works when i create the index. But failed when I want to update the b and k1 without reindexing.

In 2.3.4 i just need to stop, update the b & k1 values then start the index again. But this routine did not work on 5.1.

I need to do this to tune the b & k1 values. reindex is not an option as it will take forever

Thank you

[cbuescher (Christoph)]

Hi,

after closing the index, I was able to update the custom similarity using:

PUT /index/_settings
{
    "similarity": {
      "my_bm25": {
        "type": "BM25",
        "b": 0.76,
        "k1": 1.8
      }
    }
}

using another k1 value. After reopeing it looks like the settings are taking effect. I'm not sure how BM25 works and whether these values affect anything for documents that have already been indexed, but I guess thats not the case. Something else must be missing on your side I guess.

[Jimmy_Oentung]

Ahaaa,
I removed the mapping section and it works like a charm.

Thank you very much Christoph

Regards,
Jimmy