# Word\_delimiter and catenate\_all doesnt work?

**URL:** <https://discuss.elastic.co/t/word-delimiter-and-catenate-all-doesnt-work/43377>\
**Category:** Elasticsearch\
**Created:** [March 3, 2016, 12:18pm UTC](https://discuss.elastic.co/t/word-delimiter-and-catenate-all-doesnt-work/43377 "2016-03-03T12:18:02Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![batmaci](https://avatars.discourse-cdn.com/v4/letter/b/f08c70/32.png) [@batmaci](https://discuss.elastic.co/u/batmaci)\
**Post date:** [March 3, 2016, 12:18pm UTC](https://discuss.elastic.co/t/word-delimiter-and-catenate-all-doesnt-work/43377/1 "2016-03-03T12:18:02Z")

</div>

I have my mapping as below

{  
"state": "open",  
"settings": {  
"index": {  
"creation\_date": "1457004137507",  
"analysis": {  
"filter": {  
"my\_word\_delimiter": {  
"catenate\_all": "true",  
"split\_on\_numerics": "true",  
"split\_on\_case\_change": "true",  
"type": "word\_delimiter",  
"preserve\_original": "true"  
}  
},  
"analyzer": {  
"my\_analyzer": {  
"filter": [  
"standard"  
,  
"lowercase"  
,  
"my\_word\_delimiter"  
],  
"type": "custom",  
"tokenizer": "whitespace"  
}  
}  
},  
"number\_of\_shards": "5",  
"number\_of\_replicas": "1",  
"uuid": "zAT\_MukSSTyIVKXQz-7YKw",  
"version": {  
"created": "2020099"  
}  
}  
},  
"mappings": {  
"Product": {  
"properties": {  
"ProductCode": {  
"index": "not\_analyzed",  
"type": "string"  
},  
"id": {  
"index": "no",  
"store": true,  
"type": "integer"  
},  
"Name": {  
"store": true,  
"type": "string"  
},  
"ShortDescription": {  
"type": "string"  
}  
}  
}

I have product with name "Brother TN-2000 Toner Black" and when I use following query with "tn 2000" or "tn-2000", I am getting it in the search result. but when I use "tn2000", it will not return me anything. I though that word\_delimiter and catenate\_all should give me expected inverted index. what am I doing wrong? can you please help me?

{"query":{"bool":{"should":[{"multi\_match":{"type":"best\_fields","query":"tn 2000","fields":["Name^7","ShortDescription^6"]}}]}}}

when I check the analyzer with the following curl query,  
curl -XGET "localhost:9200/myIndex/\_analyze?analyzer=my\_analyzer&pretty  
=true" -d 'Brother TN-2000 Toner Black'

it returns me

{  
"tokens" : [ {  
"token" : "'brother",  
"start\_offset" : 0,  
"end\_offset" : 8,  
"type" : "word",  
"position" : 0  
}, {  
"token" : "brother",  
"start\_offset" : 1,  
"end\_offset" : 8,  
"type" : "word",  
"position" : 0  
} ]  
}  
curl: (6) Could not resolve host: TN-2000  
curl: (6) Could not resolve host: Toner  
curl: (6) Could not resolve host: Black'

---

<div class="post-metadata">

**Author:** ![jimczi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jimczi/32/47985_2.png) [@jimczi](https://discuss.elastic.co/u/jimczi)\
**Post date:** [March 3, 2016, 2:56pm UTC](https://discuss.elastic.co/t/word-delimiter-and-catenate-all-doesnt-work/43377/2 "2016-03-03T14:56:43Z")

</div>

I tried your example and it works for me:

```auto
curl -XGET "localhost:9200/test/_analyze?analyzer=my_analyzer&pretty=true" -d 'Brother TN-2000 Toner Black'
{
  "tokens" : [ {
    "token" : "brother",
    "start_offset" : 0,
    "end_offset" : 7,
    "type" : "word",
    "position" : 0
  }, {
    "token" : "tn-2000",
    "start_offset" : 8,
    "end_offset" : 15,
    "type" : "word",
    "position" : 1
  }, {
    "token" : "tn",
    "start_offset" : 8,
    "end_offset" : 10,
    "type" : "word",
    "position" : 1
  }, {
    "token" : "tn2000",
    "start_offset" : 8,
    "end_offset" : 15,
    "type" : "word",
    "position" : 1
  }, {
    "token" : "2000",
    "start_offset" : 11,
    "end_offset" : 15,
    "type" : "word",
    "position" : 2
  }, {
    "token" : "toner",
    "start_offset" : 16,
    "end_offset" : 21,
    "type" : "word",
    "position" : 3
  }, {
    "token" : "black",
    "start_offset" : 22,
    "end_offset" : 27,
    "type" : "word",
    "position" : 4
  } ]
}

```

Are you sure that you copy/paste the curl command you've executed ?  
The problem seems related to your curl command (could not resolve host...), try using double quotes instead ?

The field 'Name' is not using your custom analyzer, is it intended ?  
Could you also indent your example correctly.

---

<div class="post-metadata">

**Author:** ![batmaci](https://avatars.discourse-cdn.com/v4/letter/b/f08c70/32.png) [@batmaci](https://discuss.elastic.co/u/batmaci)\
**Post date:** [March 3, 2016, 3:19pm UTC](https://discuss.elastic.co/t/word-delimiter-and-catenate-all-doesnt-work/43377/3 "2016-03-03T15:19:31Z")

</div>

yea, double quotes made difference and i got results like you with curl but it still didnt solve my main problem. So this curl query shows me that in my inverted index and all searches are executed in inverted index right? if I have tn2000 and my multimatch query above should find the tn2000 in the inverted index but why It doesnt work? Do you have any idea?

when I use head plugin and under Actions menu-\> Test Analyzer, I type Brother TN-2000 Toner Black and it will return me only brother, tn, 2000, toner and black but not tn2000 as shown in the image. it looks like it uses standart analyzer

![](https://us1.discourse-cdn.com/elastic/original/2X/2/2587ec759a9dd74e9915fc961a2e2a16634dde53.png)

---

<div class="post-metadata">

**Author:** ![jimczi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jimczi/32/47985_2.png) [@jimczi](https://discuss.elastic.co/u/jimczi)\
**Post date:** [March 3, 2016, 3:23pm UTC](https://discuss.elastic.co/t/word-delimiter-and-catenate-all-doesnt-work/43377/4 "2016-03-03T15:23:13Z")

</div>

Yes it's because you don't map your field with your analyzer. You should define your field like this:

```auto
"Name": {
  "store": true,
  "type": "string",
  "index_analyzer": "my_analyzer"
},

```

You'll need to reindex your data to see the changes...

---

<div class="post-metadata">

**Author:** ![batmaci](https://avatars.discourse-cdn.com/v4/letter/b/f08c70/32.png) [@batmaci](https://discuss.elastic.co/u/batmaci)\
**Post date:** [March 3, 2016, 4:54pm UTC](https://discuss.elastic.co/t/word-delimiter-and-catenate-all-doesnt-work/43377/6 "2016-03-03T16:54:37Z")

</div>

This is how I attempted it

{  
"Product": {  
"\_source": {  
"enabled": true  
},  
"properties": {  
"id": {  
"store": true,  
"index": "no",  
"type": "integer"  
},  
"Name": {  
"index\_analyzer": "my\_analyzer",  
"type": "string"  
},  
"ShortDescription": {  
"index\_analyzer": "my\_analyzer",  
"type": "string"  
}  
}  
}  
}

but it returns me http 400 with following message

{"error":{"root\_cause":[{"type":"mapper\_parsing\_exception","reason":"Mapping definition for [Name] has unsupported parameters: [index\_analyzer : my\_analyzer]"}],"type":"mapper\_parsing\_exception","reason":"Mapping definition for [Name] has unsupported parameters: [index\_analyzer : my\_analyzer]"},"status":400}

---

<div class="post-metadata">

**Author:** ![batmaci](https://avatars.discourse-cdn.com/v4/letter/b/f08c70/32.png) [@batmaci](https://discuss.elastic.co/u/batmaci)\
**Post date:** [March 3, 2016, 5:04pm UTC](https://discuss.elastic.co/t/word-delimiter-and-catenate-all-doesnt-work/43377/7 "2016-03-03T17:04:15Z")

</div>

analyzer instead of using index\_analyzer as below worked fine. what is the difference?

```
  "Name": {
    "analyzer": "my_analyzer",
    "type": "string"
  },
```

---

<div class="post-metadata">

**Author:** ![jimczi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jimczi/32/47985_2.png) [@jimczi](https://discuss.elastic.co/u/jimczi)\
**Post date:** [March 3, 2016, 5:05pm UTC](https://discuss.elastic.co/t/word-delimiter-and-catenate-all-doesnt-work/43377/8 "2016-03-03T17:05:23Z")

</div>

Yes sorry you need to define a search\_analyzer and an analyzer for this.

````auto
"Name": {
  "store": true,
  "type": "string",
  "analyzer": "my_analyzer",
  "search_analyzer": "standard"
}
```
````

---

<div class="post-metadata">

**Author:** ![batmaci](https://avatars.discourse-cdn.com/v4/letter/b/f08c70/32.png) [@batmaci](https://discuss.elastic.co/u/batmaci)\
**Post date:** [March 4, 2016, 11:04am UTC](https://discuss.elastic.co/t/word-delimiter-and-catenate-all-doesnt-work/43377/9 "2016-03-04T11:04:35Z")

</div>

it worked like a charm. thanks for your help.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 11:11pm UTC](https://discuss.elastic.co/t/word-delimiter-and-catenate-all-doesnt-work/43377/10 "2017-07-05T23:11:17Z")

</div>