Unknown filter type [asciifolding]


(Daria Nikiforova) #1

Hello,

Mapping template contains custom analyzer that uses "asciifolding" filter and I have unit tests for this mapping. Everything worked fine before the update of Elasticsearch Test Framework version from 5.2.0 to 6.2.3. Now tests fail with the following error:

java.lang.IllegalArgumentException: Unknown filter type [asciifolding] for [asciifolding_with_preserve]

	at __randomizedtesting.SeedInfo.seed([59B1D4A9911E486:DD542AF56C53EE89]:0)
	at org.elasticsearch.index.analysis.AnalysisRegistry.getAnalysisProvider(AnalysisRegistry.java:391)
	at org.elasticsearch.index.analysis.AnalysisRegistry.buildMapping(AnalysisRegistry.java:341)

Settings are stored in a separate file "index-settings.json" with the following content:
{
"index": {
"refresh_interval": "60s",
"number_of_shards" : NUMBER_OF_SHARDS,
"number_of_replicas" : 1
},
"analysis": {
"normalizer": {
"lowercase_normalizer": {
"type": "custom",
"char_filter": [],
"filter": [
"lowercase"
]
}
},
"analyzer": {
"keywords_analyzer": {
"type": "custom",
"tokenizer": "standard",
"filter": [
"standard",
"lowercase",
"asciifolding_with_preserve"
]
}
},
"filter" : {
"asciifolding_with_preserve" : {
"type" : "asciifolding",
"preserve_original" : true
}
}
}
}

And here is a configuration:

public interface EsTestConfiguration {

static void prepareMapping(Client esClient, String indexName) {
    InputStream settingsResource = Placeholder.class.getResourceAsStream("/index-settings.json");
    String settings = new BufferedReader(new InputStreamReader(settingsResource)).lines()
            .collect(Collectors.joining("\n"));

    if (!esClient.admin().indices().prepareExists(indexName).get().isExists()) {
        esClient.admin().indices().prepareCreate(indexName).setSettings(settings, XContentType.JSON).get();
    }
    InputStream mappingsResource = Placeholder.class.getResourceAsStream("/mapping.json");

    String json = new BufferedReader(new InputStreamReader(mappingsResource)).lines()
            .parallel().collect(Collectors.joining("\n"));

    esClient.admin().indices().preparePutMapping(indexName)
            .setType("some-type")
            .setSource(json, XContentType.JSON)
            .get();
}

}

Also I noticed that tests are passed successfully if "asciifolding" is changed on another filter, for example "stop". Documentation for the ES 6.2.3 claims that "asciifolding" filter exists there..
Could you please help with this issue?


(Alexander Reelsen) #2

can you include the full stack trace?

Does this also not work, when you create the index in the dev tools console?


(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.