Why doesn't this Synonym work?


(Lee Salter) #1

Ok, so I created my index with the following:

PUT product
{
  "settings": {
    "analysis": {
      "char_filter": {
        "&_to_and": {
          "type": "mapping",
          "mappings": [
            "& => and"
          ]
        }
      },
      "filter": {
        "productSynonyms":{
          "type": "synonym",
          "synonyms" : [
            "kfc,kentucky fried chicken",
            "dairy,milk",
            "dairy,cheese",
            "dairy,cream",
            "dairy,butter"
            ]
        }
      }, 
      "analyzer": {
        "productsAnalyzer": {
          "tokenizer": "standard",
          "filter": [
            "lowercase",
            "stop",
            "snowball"
          ]
        }
      }
    }
  }
}

However, issuing the following search brings no results.

GET product/_search
{
  "query": {
    "match": {
      "brandName": "kfc"
    }
  }
}

But, the following does:

GET product/_search
{
  "query": {
    "match": {
      "brandName": "kentucky fried chicken"
    }
  }
}

Any ideas why this might be?
TIA.


(David Pilato) #2

What is your mapping?


(Lee Salter) #3

Hi David,

I had let elastic take care of the mapping automatically when indexing the documents. Brandname is mapped to a string.


(David Pilato) #4

So it uses the standard analyzer. Not yours I guess.


(Lee Salter) #5

Ah, yes I suppose that's right. So I'd need to specify the mapping to use my analyzer at index creation time??


(Jun Ohtani) #6

And you didn't set productSynonyms to productsAnalyzer.


(Lee Salter) #7

Ok, thanks for that Jun. As you might have guessed, I'm new to Elastic!


(Lee Salter) #8

Ok, so I've dropped and created the index (this is only on Test so I can do that!) with the mapping set to use the correct analyzer and the productSynonyms using productsAnalyzer.

However, issuing a search for kfc in the brandName brings back results where the word Chicken appears, not the entire phrase Kentucky Fried Chicken, if that makes sense?

How do I configure the analyzer/synonyms/mappings to work in this way?


(David Pilato) #9

Ideally send a full recreation script as explained here so we can play with it and help you fixing it.

And please format your code using </> icon. It will make your post more readable.
I updated your original post.


(Lee Salter) #10

Ok, here's my script. Thanks for your help.

DELETE productstest
PUT productstest
{
  "settings": {
    "analysis": {
      "char_filter": {
        "&_to_and": {
          "type": "mapping",
          "mappings": [
            "& => and"
          ]
        }
      },
      "filter": {
        "productSynonyms":{
          "type": "synonym",
          "synonyms" : [
            "kfc,kentucky fried chicken"
            ]
        }
      }, 
      "analyzer": {
        "productsAnalyzer": {
          "tokenizer": "standard",
          "filter": [
            "lowercase",
            "stop",
            "snowball",
            "productSynonyms"
          ]
        }
      }
    }
  }
}

PUT /productstest/_mapping/*?update_all_types
{
  "properties":{
    "brandName":{
      "type":"string",
      "analyzer":"productsAnalyzer"
    }
  }
}

POST productstest/product
{
  "brandName": "Kentucky Fried Chicken",
  "productFullName": "Zinger Tower Burger"
}

POST productstest/product
{
  "brandName": "Chicken Tonight",
  "productFullName":"Sweet and Sour Sauce"
}

POST productstest/product
{
  "brandName": "Fried dot com",
  "productFullName":"Refried beans from Mexico"
}

GET productstest/_search
{
  "query": {
    "match": {
      "brandName": "kfc"
    }
  }
}

(Jun Ohtani) #11

In you case, you should move productSynonyms filter before stop filter in you analyzer setting.

...
"filter": [
  "lowercase",
  "productSynonyms",
  "stop", 
  "snowball"
...

"Kentucky Fried Chicken" isn't expanded to "kfc" because snowball synonym filter change it to "kentucki fri chicken".

You can see how to work the analysis chain using analyze API.
example :


POST /productstest/_analyze
{
  "text": "Kentucky Fried Chicken",
  "field": "brandName"
}

See docs : Synonyms and analysis chain


(Lee Salter) #12

Thanks for your help on this Jun. I've moved the filter up the chain, but still get the Chicken Tonight product back when searching for kfc - is there any way to only return the Kentucky Fried Chicken product when searching for kfc?

I will read up on this via the link you sent. Again, thanks for your input.


(Lee Salter) #13

Figured it out!!
I needed to use a uni-directional synonym like this:

PUT productstest
{
  "settings": {
    "analysis": {
      "char_filter": {
        "&_to_and": {
          "type": "mapping",
          "mappings": [
            "& => and"
          ]
        }
      },
      "filter": {
        "productSynonyms": {
          "type": "synonym",
          "synonyms": [
            "kentucky fried chicken=>kfc"
          ]
        }
      },
      "analyzer": {
        "productsAnalyzer": {
          "tokenizer": "standard",
          "filter": [
            "lowercase",
            "productSynonyms",
            "stop"
          ]
        }
      }
    }
  }
}

Now, the query:

GET productstest/_search
{
  "query": {
    "match": {
      "brandName": "kfc"
    }
  }
}

returns only the Kentucky Fried Chicken result.


(system) #14