# Hyphenation decompounder token filter seems to ignore the \`only\_longest\_match\` option

**URL:** https://discuss.elastic.co/t/hyphenation-decompounder-token-filter-seems-to-ignore-the-only-longest-match-option/294047
**Category:** Elasticsearch
**Created:** [January 11, 2022, 2:31pm UTC](https://discuss.elastic.co/t/hyphenation-decompounder-token-filter-seems-to-ignore-the-only-longest-match-option/294047 "2022-01-11T14:31:23Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![Jelena\_Malinovic](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jelena_malinovic/32/100125_2.png) [@Jelena\_Malinovic](https://discuss.elastic.co/u/Jelena_Malinovic)
#### Post date: [January 11, 2022, 2:31pm UTC](https://discuss.elastic.co/t/hyphenation-decompounder-token-filter-seems-to-ignore-the-only-longest-match-option/294047/1 "2022-01-11T14:31:23Z")

</div>

Hi everyone,

We are trying to improve searches on our website which include compounds words. For that purpose, we are using the token filter `hyphenation_decompounder`. During testing, we found out that `only_longest_match` option doesn't work as intended.

Here is the output after running the `_analyze` API. Of course, we used a much longer list of words, but I have singled out here only words relevant to my example.

```auto
GET _analyze
{
  "tokenizer": "whitespace",
  "filter": [
    {
      "type": "hyphenation_decompounder",
      "hyphenation_patterns_path" : "german_hyphenation_patterns.xml",
      "only_longest_match": true,
      "max_subword_size": 22,
      "word_list": [
        "kinder",
        "wagen",
        "gen"
      ]
    }
  ],
  "text": "kinderwagen"
}

```

The result is:

```auto
{
  "tokens" : [
    {
      "token" : "kinderwagen",
      "start_offset" : 0,
      "end_offset" : 11,
      "type" : "word",
      "position" : 0
    },
    {
      "token" : "kinder",
      "start_offset" : 0,
      "end_offset" : 11,
      "type" : "word",
      "position" : 0
    },
    {
      "token" : "wagen",
      "start_offset" : 0,
      "end_offset" : 11,
      "type" : "word",
      "position" : 0
    },
    {
      "token" : "gen",
      "start_offset" : 0,
      "end_offset" : 11,
      "type" : "word",
      "position" : 0
    }
  ]
}

```

It seems like "gen" is still considered as a relevant token for this case, although a words lists contain "wagen" and we set `only_longest_match` to true.

Any ideas why this happens? Are we missing something here, in the token filter configuration? Can this be a bug?

Any help is appreciated.

---

<div class="post-metadata">

### Author: ![Tomo\_M](https://avatars.discourse-cdn.com/v4/letter/t/848f3c/32.png) [@Tomo\_M](https://discuss.elastic.co/u/Tomo_M)
#### Post date: [January 11, 2022, 3:26pm UTC](https://discuss.elastic.co/t/hyphenation-decompounder-token-filter-seems-to-ignore-the-only-longest-match-option/294047/2 "2022-01-11T15:26:18Z")

</div>

Hi,

I found a relevant discussion. Unfortunately, it seems a known Lucene behaviour. If so, the name 'only\_longest\_match' is a bit confusing but consistent with Lucene onlyLongedtMatch flag.

> [@Compound word token filter: only\_longest\_match doesn't work as expected in some scenarios](https://discuss.elastic.co/t/compound-word-token-filter-only-longest-match-doesnt-work-as-expected-in-some-scenarios/195470/2):
>
> The only\_longest\_match option may not quite work how you think it does. [This Lucene issue comment](https://issues.apache.org/jira/browse/LUCENE-8183?focusedCommentId=16374886&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16374886) has details on how it works: The onlyLongestMatch flag currently affects whether all matches or only the longest match should be returned per start character (in DictionaryCompoundWordTokenFilter) or per hyphenation start point (in HyphenationCompoundWordTokenFilter). Example: Dictionary "Schaft", "Wirt", "Wirtschaft", "Wissen", "Wissenschaft" for input "Wirtschaftswissenschaft" will retu…

---

<div class="post-metadata">

### Author: ![Jelena\_Malinovic](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jelena_malinovic/32/100125_2.png) [@Jelena\_Malinovic](https://discuss.elastic.co/u/Jelena_Malinovic)
#### Post date: [January 12, 2022, 5:12pm UTC](https://discuss.elastic.co/t/hyphenation-decompounder-token-filter-seems-to-ignore-the-only-longest-match-option/294047/3 "2022-01-12T17:12:53Z")

</div>

Thanks for clarifying this!

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 9, 2022, 5:13pm UTC](https://discuss.elastic.co/t/hyphenation-decompounder-token-filter-seems-to-ignore-the-only-longest-match-option/294047/4 "2022-02-09T17:13:00Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
