# Synonym Token Filter Error

**URL:** <https://discuss.elastic.co/t/synonym-token-filter-error/160016>\
**Category:** Elasticsearch\
**Created:** [December 8, 2018, 7:57pm UTC](https://discuss.elastic.co/t/synonym-token-filter-error/160016 "2018-12-08T19:57:32Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jensxy](https://avatars.discourse-cdn.com/v4/letter/j/ea666f/32.png) [@Jensxy](https://discuss.elastic.co/u/Jensxy)\
**Post date:** [December 8, 2018, 7:57pm UTC](https://discuss.elastic.co/t/synonym-token-filter-error/160016/1 "2018-12-08T19:57:32Z")

</div>

Synonym Token Filter gives an Error.

**Elasticsearch version**  
Version: 6.2.2, Build: 10b1edd/2018-02-16T19:01:30.685723Z, JVM: 1.8.0\_161

**Plugins installed** :  
Opennlp

**JVM version**  
java version "1.8.0\_161"  
Java(TM) SE Runtime Environment (build 1.8.0\_161-b12)  
Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)

**OS version**  
Linux server 4.4.0-131-generic #157-Ubuntu SMP Thu Jul 12 15:51:36 UTC 2018 x86\_64 x86\_64 x86\_64 GNU/Linux

**Description of the problem including expected versus actual behavior** :

When I try to reproduce the following: [https://qbox.io/blog/synonym-token-filter-wordnet-applications](https://qbox.io/blog/synonym-token-filter-wordnet-applications)

I get the following error.

Error: 400 - failed to build synonyms  
ES stack trace:

type: illegal\_argument\_exception  
reason: failed to build synonyms

**Steps to reproduce** :

```auto
curl -XPUT 'localhost:9200/test_index' -d '{
 "analysis": {
    "filter": {
      "synonym": {
        "type": "synonym",
        "format": "wordnet",
        "synonyms_path": "analysis/wn_s.pl"
      }
    },
    "analyzer": {
      "wordnet-synonym-analyzer": {
        "tokenizer": "lowercase",
        "filter": [
          "synonym"
        ]
      }
    }
  }
}'

```

**EDIT:**

If I change the tokenizer to "standard" everything is fine. Does anyone have an idea what to do?

---

<div class="post-metadata">

**Author:** ![abdon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abdon/32/9195_2.png) [@abdon](https://discuss.elastic.co/u/abdon)\
**Post date:** [December 9, 2018, 11:01am UTC](https://discuss.elastic.co/t/synonym-token-filter-error/160016/2 "2018-12-09T11:01:17Z")

</div>

The problem is with line 40602 in your synonym file:

```auto
s(104178190,2,'78',n,2,0).

```

The number 78 is completely removed by your lowercase tokenizer. That tokenizer is based on the letter tokenizer which has this behavior.

There are a few options to solve this issue. Firstly, you could remove the synonyms that are pure numbers from your synonym file, like line 40602 in `wn_s.pl`.

Or, you could switch to an analyzer that does not drop numbers. For example the standard analyzer in combination with the lowercase token filter (probably the best solution):

```auto
PUT /analyzers-blog-04-02
{
 "analysis": {
    "filter": {
      "synonym": {
        "type": "synonym",
        "format": "wordnet",
        "synonyms_path": "analysis/wn_s.pl"
      }
    },
    "analyzer": {
      "wordnet-synonym-analyzer": {
        "tokenizer": "standard",
        "filter": [
          "lowercase",
          "synonym"
        ]
      }
    }
  }
}

```

Or, if you really want to use the existing analyzer, you can set `"lenient": true` in your synonym token filter definition:

```auto
PUT /analyzers-blog-04-02
{
 "analysis": {
    "filter": {
      "synonym": {
        "type": "synonym",
        "format": "wordnet",
        "synonyms_path": "analysis/wn_s.pl",
        "lenient": true
      }
    },
    "analyzer": {
      "wordnet-synonym-analyzer": {
        "tokenizer": "lowercase",
        "filter": [
          "synonym"
        ]
      }
    }
  }
}

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 6, 2019, 11:01am UTC](https://discuss.elastic.co/t/synonym-token-filter-error/160016/3 "2019-01-06T11:01:25Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.