I was using a custom analyzer with two filters: lowercase + asciifolding.
It doesn't work anymore in 6.0.0 and 6.0.1.
The easiest way to reproduce the problem:
POST /_analyze
{
"tokenizer": "standard",
"filter": [ "lowercase", "asciifolding" ],
"text": "Is this déja vu?"
}
Result:
{
"error": {
"root_cause": [
{
"type": "remote_transport_exception",
"reason": "[pyJpaTt][127.0.0.1:9300][indices:admin/analyze[s]]"
}
],
"type": "illegal_argument_exception",
"reason": "failed to find global token filter under [asciifolding]"
},
"status": 400
}
If I remove the asciifolding filter:
{
"tokens": [
{
"token": "is",
"start_offset": 0,
"end_offset": 2,
"type": "<ALPHANUM>",
"position": 0
},
{
"token": "this",
"start_offset": 3,
"end_offset": 7,
"type": "<ALPHANUM>",
"position": 1
},
{
"token": "déja",
"start_offset": 8,
"end_offset": 12,
"type": "<ALPHANUM>",
"position": 2
},
{
"token": "vu",
"start_offset": 13,
"end_offset": 15,
"type": "<ALPHANUM>",
"position": 3
}
]
}
Has asciifolding been removed? Is it available via a plugin? Should I use something else?