Stopword with Query_string


(vgross) #1

Hi !

I have a problem with the stopword when I make a request of type
"query_string".

Here my index creation :

{
"number_of_shards":5,
"number_of_replicas":1,
"analysis":{
"analyzer":{
"indexAnalyzerAc":{
"type":"custom",
"tokenizer":"whitespace",
"filter":[

           "synonym",
           "lowercase",
           "asciifolding"
        ]
     },
     "searchAnalyzerAc":{
        "type":"custom",
        "tokenizer":"whitespace",
        "filter":[

           "synonym",
           "lowercase",
           "asciifolding"
        ]
     },
     "indexAnalyzerProd":{
        "type":"custom",
        "tokenizer":"whitespace",
        "filter":[

           "synonym",
           "lowercase",
           "asciifolding"
        ]
     },
     "searchAnalyzerProd":{
        "type":"custom",
        "tokenizer":"whitespace",
        "filter":[
           "synonym",
           "lowercase",
           "asciifolding",
                          "mystop"
        ]
     }
  },
  "filter":{
     "synonym":{
        "type":"synonym",
        "synonyms_path":"\/etc\/elasticsearch\/synonym.txt"
     },
     "mystop":{
        "type":"stop",
        "stopwords":[
           "_french_"
        ],
        "ignore_case":"true"
     }
  }

}
}

And Here my Query :

{
"query":{
"query_string":{
"query":"robes en rouge",
"default_field":"_all",
"default_operator":"and",
"fields":[
"_all^1.0",
"matiere^1",
"tailles.nom1",
"couleurs^1",
"description^1",
"ref_mark^1",
"libelle^1",
"marque^1",
"scat^2",
"cat^3",
"genre^1"
]
}
}
}

The problem is when I search "robes en rouge ", "en" being a stopword, if
the filter is activated, the search returns any results to me, by cons, if
the filter is disabled, I have 135 results.
If I make the search "robes rouge" I have 175 results, the aim being to
have the same number of result when searching "robe en rouge". But i have 0
résult :frowning:

may be that the stop word filter is not activated with this type of request?

Can you help me ?

Thanks.


(vgross) #2

nobody can help me ? :frowning:

Le lundi 18 juin 2012 10:40:59 UTC+2, Vincent GROSS a écrit :

Hi !

I have a problem with the stopword when I make a request of type
"query_string".

Here my index creation :

{
"number_of_shards":5,
"number_of_replicas":1,
"analysis":{
"analyzer":{
"indexAnalyzerAc":{
"type":"custom",
"tokenizer":"whitespace",
"filter":[

           "synonym",
           "lowercase",
           "asciifolding"
        ]
     },
     "searchAnalyzerAc":{
        "type":"custom",
        "tokenizer":"whitespace",
        "filter":[

           "synonym",
           "lowercase",
           "asciifolding"
        ]
     },
     "indexAnalyzerProd":{
        "type":"custom",
        "tokenizer":"whitespace",
        "filter":[

           "synonym",
           "lowercase",
           "asciifolding"
        ]
     },
     "searchAnalyzerProd":{
        "type":"custom",
        "tokenizer":"whitespace",
        "filter":[
           "synonym",
           "lowercase",
           "asciifolding",
                          "mystop"
        ]
     }
  },
  "filter":{
     "synonym":{
        "type":"synonym",
        "synonyms_path":"\/etc\/elasticsearch\/synonym.txt"
     },
     "mystop":{
        "type":"stop",
        "stopwords":[
           "_french_"
        ],
        "ignore_case":"true"
     }
  }

}
}

And Here my Query :

{
"query":{
"query_string":{
"query":"robes en rouge",
"default_field":"_all",
"default_operator":"and",
"fields":[
"_all^1.0",
"matiere^1",
"tailles.nom1",
"couleurs^1",
"description^1",
"ref_mark^1",
"libelle^1",
"marque^1",
"scat^2",
"cat^3",
"genre^1"
]
}
}
}

The problem is when I search "robes en rouge ", "en" being a stopword, if
the filter is activated, the search returns any results to me, by cons, if
the filter is disabled, I have 135 results.
If I make the search "robes rouge" I have 175 results, the aim being to
have the same number of result when searching "robe en rouge". But i have 0
résult :frowning:

may be that the stop word filter is not activated with this type of
request?

Can you help me ?

Thanks.


V5.1 query_string vs v2.4 query_string concerning to stopwords
(vgross) #3

I just found the solution thanks to a french group of Elasticsearch!

In fact, if I explicitly set my analyzer when I make the request, good
analyzer is considered and the application works.

It works for my request is as follows:

{
"query":{
"query_string":{
"query":"robes en rouge",
"default_field":"_all",
"default_operator":"and",
"analyzer" : "searchAnalyzerProd"
"fields":[
"_all^1.0",
"matiere^1",
"tailles.nom1",
"couleurs^1",
"description^1",
"ref_mark^1",
"libelle^1",
"marque^1",
"scat^2",
"cat^3",
"genre^1"
]
}
}
}

Thank to Benjamin Levient for his help !

Le lundi 18 juin 2012 10:40:59 UTC+2, Vincent GROSS a écrit :

Hi !

I have a problem with the stopword when I make a request of type
"query_string".

Here my index creation :

{
"number_of_shards":5,
"number_of_replicas":1,
"analysis":{
"analyzer":{
"indexAnalyzerAc":{
"type":"custom",
"tokenizer":"whitespace",
"filter":[

           "synonym",
           "lowercase",
           "asciifolding"
        ]
     },
     "searchAnalyzerAc":{
        "type":"custom",
        "tokenizer":"whitespace",
        "filter":[

           "synonym",
           "lowercase",
           "asciifolding"
        ]
     },
     "indexAnalyzerProd":{
        "type":"custom",
        "tokenizer":"whitespace",
        "filter":[

           "synonym",
           "lowercase",
           "asciifolding"
        ]
     },
     "searchAnalyzerProd":{
        "type":"custom",
        "tokenizer":"whitespace",
        "filter":[
           "synonym",
           "lowercase",
           "asciifolding",
                          "mystop"
        ]
     }
  },
  "filter":{
     "synonym":{
        "type":"synonym",
        "synonyms_path":"\/etc\/elasticsearch\/synonym.txt"
     },
     "mystop":{
        "type":"stop",
        "stopwords":[
           "_french_"
        ],
        "ignore_case":"true"
     }
  }

}
}

And Here my Query :

{
"query":{
"query_string":{
"query":"robes en rouge",
"default_field":"_all",
"default_operator":"and",
"fields":[
"_all^1.0",
"matiere^1",
"tailles.nom1",
"couleurs^1",
"description^1",
"ref_mark^1",
"libelle^1",
"marque^1",
"scat^2",
"cat^3",
"genre^1"
]
}
}
}

The problem is when I search "robes en rouge ", "en" being a stopword, if
the filter is activated, the search returns any results to me, by cons, if
the filter is disabled, I have 135 results.
If I make the search "robes rouge" I have 175 results, the aim being to
have the same number of result when searching "robe en rouge". But i have 0
résult :frowning:

may be that the stop word filter is not activated with this type of
request?

Can you help me ?

Thanks.


(system) #4