Strange behaviour when i fetch data from elastic

Elastic Stack Elasticsearch
1

My dataset in elastic looks like this.

Category Scat Sscat Products Measure Price
Beverages Soft Drinks Cans Pepsi My Soft Drink 250Ml 250Ml 30
Household Needs Laundry Detergents Detergent Powders Ariel Matic Top Load Detergent Powder 2Kg 2Kg 449
Household Needs Dishwashers Scrubbers Gala Sponge Wipe 5 Units 5 Units 195
Personal Care Hair Care Shampoo Ayush Anti Dandruff Neem Shampoo 330Ml 330Ml 199

I have mapped data to elastic in following way-

PUT _template/template_1
{
    "index_patterns": [
      "ubq-*"
    ],
    "settings": {
      "index": {
        "analysis": {
          "filter": {
            "whitespace_remove": {
              "pattern": " ",
              "type": "pattern_replace",
              "replacement": ""
            }
          },
          "analyzer": {
            "my_analyzer": {
              "filter": [
                "lowercase",
                "whitespace_remove",
                "nGram"
              ],
              "type": "custom",
              "tokenizer": "keyword"
            }
          }
        },
        "number_of_shards": "1"
      }
    },
    "mappings": {
      "doc": {
        "properties": {
          "Index": {
            "type": "float",
            "index": "true"
          },
          "Category": {
            "type": "text",
            "index": "true",
            "analyzer": "my_analyzer"
          },
          "Scat": {
            "type": "text",
            "index": "true",
            "analyzer": "my_analyzer"
          },
          "Sscat": {
            "type": "text",
            "index": "true",
            "analyzer": "my_analyzer"
          },
          "Products": {
            "type": "text",
            "index": "true",
            "analyzer": "my_analyzer"
          },
          "Measure": {
            "type": "text",
            "index": "true",
            "analyzer": "my_analyzer"
          },
          "Price": {
            "type": "float",
            "index": "true"
          },
          "Description": {
            "type": "text",
            "index": "true",
            "analyzer": "my_analyzer"
          },
          "Gst": {
            "type": "float",
            "index": "true"
          },
          "Url": {
            "type": "text",
            "index": "true",
            "analyzer": "my_analyzer"
          }
        }
      }
    },
    "aliases": {}
  }
}

I am fetching Products from elastic and displaying in their respective Sscat in my android app. Im some Sscat products displayed are fine. But in some Sscat products are being displayed that do not belong to that sscat. Can someone tell me what I might be doing wrong here?

2

Wrap code in ``` to make it more readable. Otherwise discuss throws out the indentation.

It is hard to tell what the problem is from here, but I have a guess. You declared sscat as a text field which means that searches over it are tokenized. If you are picking the category from a drop down then you probably want it as a keyword field.

Beyond that, I suspect the ngram filter is making this process quite sloppy. Using it like this is likely to cause your searches to match documents that don't exactly match. That could be useful for misspellings and stuff, but is generally unexpected. I rarely use ngram, but when I do it is with great care and generally it is part of a strategy involving analysing text multiple ways and searching for it with multiple analyzers, weighting the ngram one much lower than a more "exact" analysis chain.

3

Hi thanks nik9000 for replying. I tried removing ngram and declaring sscat filed as a keyword, but still i am not getting the desired result. Still i am fetching other products in sscats.
My new mapping looks like this.

    PUT _template/template_1
    {
        "index_patterns": [
          "ubq-*"
        ],
        "settings": {
          "index": {
            "analysis": {
              "filter": {
                "whitespace_remove": {
                  "pattern": " ",
                  "type": "pattern_replace",
                  "replacement": ""
                }
              },
              "analyzer": {
                "my_analyzer": {
                  "filter": [
                    "lowercase",
                    "whitespace_remove"
                ],
                  "type": "custom",
                  "tokenizer": "keyword"
                }
              }
            },
            "number_of_shards": "1"
          }
        },
        "mappings": {
          "doc": {
            "properties": {
              "Index": {
                "type": "float",
                "index": "true"
              },
              "Category": {
                "type": "keyword",
                "index": "true"
              },
              "Scat": {
                "type": "keyword",
                "index": "true"
              },
              "Sscat": {
                "type": "keyword",
                "index": "true"
              },
              "Products": {
                "type": "text",
                "index": "true",
                "analyzer": "my_analyzer"
              },
              "Measure": {
                "type": "text",
                "index": "true",
                "analyzer": "my_analyzer"
              },
              "Price": {
                "type": "float",
                "index": "true"
              },
              "Description": {
                "type": "text",
                "index": "true",
                "analyzer": "my_analyzer"
              },
              "Gst": {
                "type": "float",
                "index": "true"
              },
              "Url": {
                "type": "text",
                "index": "true",
                "analyzer": "my_analyzer"
              }
            }
          }
        },
        "aliases": {}
      }
    }

I am quite new to elastic sorry if this is some silly problem.

4

Just checking, you changed the template, but does the index have the new mapping? The mapping is assigned at index creation time.

At this point it might be useful to recreate your problem as a simple bash script that uses curl to do all the things. Either you'll where you are going wrong or you won't but you can share the script with us and we'll know better what you are trying to do.

5

I changed the mapping and recreated the index. Ill try 2nd option in a while and get back. Thanks.

6

Issue was fixed. Thanks

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.