Queries for stem words or synonyms dont yield results


(Apostol Marinov) #1

Mapping and settings:

{
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 0,
    "analysis": {
      "filter": {
        "stop": {
          "type": "stop",
          "stopwords": "_bulgarian_"
        },
        "stop_en": {
          "type": "stop",
          "stopwords": "_english_"
        },
        "stemmer": {
          "type": "stemmer",
          "name": "bulgarian"
        },
        "stemmer_en": {
          "type": "stemmer",
          "name": "english"
        },
        "synonym": {
          "type": "synonym",
          "tokenizer": "whitespace",
          "synonyms": [
            "стоматолози,асд"
          ]
        },
        "synonym_en": {
          "type": "synonym",
          "tokenizer": "whitespace",
          "synonyms": [
            "dentists,asd"
          ]
        }
      },
      "analyzer": {
        "filter_analyzer": {
          "type": "custom",
          "tokenizer": "standard",
          "filter": [
            "lowercase",
            "synonym",
            "stop",
            "stemmer"
          ]
        },
        "filter_analyzer_en": {
          "type": "custom",
          "tokenizer": "standard",
          "filter": [
            "lowercase",
            "synonym_en",
            "stop_en",
            "stemmer_en"
          ]
        }
      }
    }
  },
  "mapping": {
    "filter": {
      "properties": {
        "filter_label": {
          "type": "text",
          "analyzer": "filter_analyzer",
          "fielddata": true
        },
        "filter_label_en": {
          "type": "text",
          "analyzer": "filter_analyzer_en",
          "fielddata": true
        },
        "filter_id": {
          "type": "short"
        },
        "filter_name": {
          "type": "text"
        },
        "filter_param_name": {
          "type": "text"
        },
        "search_category_id": {
          "type": "short"
        }
      }
    }
  }
}

Notes on mapping: I create the index from PHP using the SDK and the mapping I provide and the one in the output from Kibana differs i.e. I've defined search_category_id and filter_id as short, but in the output mapping from Kibana they are long. Also I've added fielddata to the text fields, but in the Kibana output fielddata is not present.

Examples:
When testing out the analyzers I get correct results both for stemming and synonyms:

GET /index/_analyze
{
  "analyzer": "filter_analyzer_en",
  "text": "dentists"
}

Outputs the provided synonym and the word in its root form:

{
  "tokens": [
    {
      "token": "dentist",
      "start_offset": 0,
      "end_offset": 8,
      "type": "<ALPHANUM>",
      "position": 0
    },
    {
      "token": "asd",
      "start_offset": 0,
      "end_offset": 8,
      "type": "SYNONYM",
      "position": 0
    }
  ]
}

But when I try querying with the root form of the word or the synonym, there are no results:

GET /index/_search
{
  "query": {
    "match": {
      "filter_label_en": {
        "query":"dentist"
      }
    }
  }
}
Results:
{
  "hits": {
    "total": 0,
    "max_score": null,
    "hits": []
  }
}
GET /index/_search
{
  "query": {
    "match": {
      "filter_label_en": {
        "query":"asd"
      }
    }
  }
}
Results:
{
  "hits": {
    "total": 0,
    "max_score": null,
    "hits": []
  }
}
GET /index/_search
{
  "query": {
    "match": {
      "filter_label_en": {
        "query":"dentists"
      }
    }
  }
}
Results:
{
  "hits": {
    "total": 1,
    "max_score": 9.404876,
    "hits": [
      {
        "_index": "index",
        "_type": "filter",
        "_id": "19",
        "_score": 9.404876,
        "_source": {
          "filter_label": "Стоматолози",
          "filter_label_en": "Dentists",
          "filter_id": 28,
          "filter_name": "profileType",
          "filter_param_name": "profileType",
          "search_category_id": 101
        }
      }
    ]
  }
}

(Apostol Marinov) #2

Turns out there was a typo in the mappings parameter, I named it "mapping" :frowning:


(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.