Elasticsearch or Lucene synonyms (wordnet) Bug

svarup · December 17, 2017, 8:06am

when i was using synonym filter in elasticsearch 5.0 it was run perfect but i just upgrade to elasticsearch 6.0.1 and it gives me error i use php elasticsearch client and wordnet synonyms.

Fatal error: Uncaught Elasticsearch\Common\Exceptions\BadRequest400Exception: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"failed to build synonyms"}],"type":"illegal_argument_exception","reason":"failed to build synonyms","caused_by":{"type":"parse_exception","reason":"Invalid synonym rule at line 109","caused_by":{"type":"illegal_argument_exception","reason":"term: course of action analyzed to a token with posinc != 1"}}},"status":400}

i think that error from lucene.

github.com

apache/lucene-solr/blob/master/lucene/analysis/common/src/java/org/apache/lucene/analysis/synonym/SolrSynonymParser.java#L72


  super(dedup, analyzer);
  this.expand = expand;
}


@Override
public void parse(Reader in) throws IOException, ParseException {
  LineNumberReader br = new LineNumberReader(in);
  try {
    addInternal(br);
  } catch (IllegalArgumentException e) {
    ParseException ex = new ParseException("Invalid synonym rule at line " + br.getLineNumber(), 0);
    ex.initCause(e);
    throw ex;
  } finally {
    br.close();
  }
}


private void addInternal(BufferedReader in) throws IOException {
  String line = null;
  while ((line = in.readLine()) != null) {

svarup · December 17, 2017, 9:42am

i also use stopFilter and when i use synonymn filter befor the stopFilter then it will solve my issue.

johtani · December 18, 2017, 3:23am

Could you share your mappings and settings?

svarup · December 20, 2017, 5:06pm

Yes here it is the mapping of synonyms

icu_synonym' => [
'type'		=> 'custom',
'char_filter'	=> ['html_strip', 'quotes'],
'tokenizer'	=> 'icu_tokenizer',
'filter'	=> ['lowercase', 'icu_normalizer', 'synonym', 'en_stop_filter', 'en_stem_filter', 'icu_folding'],
],

johtani · December 25, 2017, 11:16pm

Can you also share each tokenizer/token filter/ char filter settings?

system · January 22, 2018, 11:16pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.