Synonym token filter question

[nages]

I created a test index with synonym token filter

> PUT /synonyms-index

{

"settings": {

"analysis": {

"filter": {

"my_synonym_filter": {

"type": "synonym",

"synonyms": [

"shares","equity","stock"

]

}

},

"analyzer": {

"my_synonyms": {

"tokenizer": "standard",

"filter": [

"lowercase",

"my_synonym_filter"

]

}

}

}

}

}

Then I ran analyze API ,
post synonyms-index/_analyze
{
"analyzer":"my_synonyms",
"text":"equity awesome"
}

I got the following response to see what token got into inverted index and I was expecting "shares" and "stock" needed to be added as per the synonym rule, but it doesn't seem so. Am I missing anything here ?

{
  "tokens": [
    {
      "token": "equity",
      "start_offset": 0,
      "end_offset": 6,
      "type": "<ALPHANUM>",
      "position": 0
    },
    {
      "token": "awesome",
      "start_offset": 7,
      "end_offset": 14,
      "type": "<ALPHANUM>",
      "position": 1
    }
  ]
}

[cbuescher (Christoph)]

@nages

"synonyms": [ "shares","equity","stock" ]

Hi @nages,

you ran into a common pitfall that I also experienced multiple times when playing with synonyms. Each rule should be its own String, so in your case "shares, equity, stock". In your current setup you have three rules which don't epxand to anything. This is easy to miss, but take a look at this snippet from our documentation to convince yourself and try it:

"synonyms" : [
        "i-pod, i pod => ipod",
        "universe, cosmos"
]

I a synonym file, which would be the usual way of managing larger collections, each line represents one rule so you don't get this problem there.

Hope this helps.

[nages]

strange . JSON thing -

I made it as "synonyms": [ "shares,equity,stock" ] instead of ("synonyms": [ "shares","equity","stock" ]) - All set