Why doesn't this Synonym work?

Elastic Stack Elasticsearch
1

Ok, so I created my index with the following:

PUT product
{
  "settings": {
    "analysis": {
      "char_filter": {
        "&_to_and": {
          "type": "mapping",
          "mappings": [
            "& => and"
          ]
        }
      },
      "filter": {
        "productSynonyms":{
          "type": "synonym",
          "synonyms" : [
            "kfc,kentucky fried chicken",
            "dairy,milk",
            "dairy,cheese",
            "dairy,cream",
            "dairy,butter"
            ]
        }
      }, 
      "analyzer": {
        "productsAnalyzer": {
          "tokenizer": "standard",
          "filter": [
            "lowercase",
            "stop",
            "snowball"
          ]
        }
      }
    }
  }
}

However, issuing the following search brings no results.

GET product/_search
{
  "query": {
    "match": {
      "brandName": "kfc"
    }
  }
}

But, the following does:

GET product/_search
{
  "query": {
    "match": {
      "brandName": "kentucky fried chicken"
    }
  }
}

Any ideas why this might be?
TIA.

2

What is your mapping?

3

Hi David,

I had let elastic take care of the mapping automatically when indexing the documents. Brandname is mapped to a string.

4

So it uses the standard analyzer. Not yours I guess.

5

Ah, yes I suppose that's right. So I'd need to specify the mapping to use my analyzer at index creation time??

6

And you didn't set productSynonyms to productsAnalyzer.

7

Ok, thanks for that Jun. As you might have guessed, I'm new to Elastic!

8

Ok, so I've dropped and created the index (this is only on Test so I can do that!) with the mapping set to use the correct analyzer and the productSynonyms using productsAnalyzer.

However, issuing a search for kfc in the brandName brings back results where the word Chicken appears, not the entire phrase Kentucky Fried Chicken, if that makes sense?

How do I configure the analyzer/synonyms/mappings to work in this way?

9

Ideally send a full recreation script as explained here so we can play with it and help you fixing it.

And please format your code using </> icon. It will make your post more readable.
I updated your original post.

10

Ok, here's my script. Thanks for your help.

DELETE productstest
PUT productstest
{
  "settings": {
    "analysis": {
      "char_filter": {
        "&_to_and": {
          "type": "mapping",
          "mappings": [
            "& => and"
          ]
        }
      },
      "filter": {
        "productSynonyms":{
          "type": "synonym",
          "synonyms" : [
            "kfc,kentucky fried chicken"
            ]
        }
      }, 
      "analyzer": {
        "productsAnalyzer": {
          "tokenizer": "standard",
          "filter": [
            "lowercase",
            "stop",
            "snowball",
            "productSynonyms"
          ]
        }
      }
    }
  }
}

PUT /productstest/_mapping/*?update_all_types
{
  "properties":{
    "brandName":{
      "type":"string",
      "analyzer":"productsAnalyzer"
    }
  }
}

POST productstest/product
{
  "brandName": "Kentucky Fried Chicken",
  "productFullName": "Zinger Tower Burger"
}

POST productstest/product
{
  "brandName": "Chicken Tonight",
  "productFullName":"Sweet and Sour Sauce"
}

POST productstest/product
{
  "brandName": "Fried dot com",
  "productFullName":"Refried beans from Mexico"
}

GET productstest/_search
{
  "query": {
    "match": {
      "brandName": "kfc"
    }
  }
}
11

In you case, you should move productSynonyms filter before stop filter in you analyzer setting.

...
"filter": [
  "lowercase",
  "productSynonyms",
  "stop", 
  "snowball"
...

"Kentucky Fried Chicken" isn't expanded to "kfc" because snowball synonym filter change it to "kentucki fri chicken".

You can see how to work the analysis chain using analyze API.
example :


POST /productstest/_analyze
{
  "text": "Kentucky Fried Chicken",
  "field": "brandName"
}

See docs : Synonyms and analysis chain

12

Thanks for your help on this Jun. I've moved the filter up the chain, but still get the Chicken Tonight product back when searching for kfc - is there any way to only return the Kentucky Fried Chicken product when searching for kfc?

I will read up on this via the link you sent. Again, thanks for your input.

13

Figured it out!!
I needed to use a uni-directional synonym like this:

PUT productstest
{
  "settings": {
    "analysis": {
      "char_filter": {
        "&_to_and": {
          "type": "mapping",
          "mappings": [
            "& => and"
          ]
        }
      },
      "filter": {
        "productSynonyms": {
          "type": "synonym",
          "synonyms": [
            "kentucky fried chicken=>kfc"
          ]
        }
      },
      "analyzer": {
        "productsAnalyzer": {
          "tokenizer": "standard",
          "filter": [
            "lowercase",
            "productSynonyms",
            "stop"
          ]
        }
      }
    }
  }
}

Now, the query:

GET productstest/_search
{
  "query": {
    "match": {
      "brandName": "kfc"
    }
  }
}

returns only the Kentucky Fried Chicken result.