Common terms query with cutoff_frequency

vinamar · February 6, 2014, 6:58pm

Hi,

I have the following query given below. Currently any query term like "email not archived" excludes the not from the search results and highlighted results. i was planning to use common terms query to include this stop word in the results.

{
"common": {
"body": {
"query": "this is bonsai cool",
"cutoff_frequency": 0.001
}
}
}

I couldn't find any example of how to do this.

Can you please let me know how should i modify my query to include this ?

{
"timeout": 30000,
"highlight": {
"pre_tags": [
""
],
"post_tags": [
""
],
"fields": {
"srId": {
"number_of_fragments": 0
},
"emails.emailBody": {
"number_of_fragments": 0
},
"chatTextArray.text": {
"number_of_fragments": 0
}
}
},
"query": {
"filtered": {
"filter": [
{
"range": {
"closeDt": {
"from": 1325404800000,
"to": 1391760000000
}
}
}
],
"query": {
"query_string": {
"query": "payment button",
"default_operator": "AND",
"fields": [
[
"emails.emailBody",
"srId",
"chatTextArray.text"
]
]
}
},
"_cache": true
}
},
"size": 10
}

spinscale · February 10, 2014, 12:44pm

Hey,

you need to replace the query_string query with a match query, which in
turn supports the cutoff_frequency. Also make sure you change the default
operator to AND. If you want to support multiple fields, you will need to
use the multi_field query

see

hope this helps...

--Alex

On Thu, Feb 6, 2014 at 7:58 PM, vinamar vinamar@gmail.com wrote:

Hi,

I have the following query given below. Currently any query term like
"email not archived" excludes the not from the search results and
highlighted results. i was planning to use common terms query to include
this stop word in the results.

{
"common": {
"body": {
"query": "this is bonsai cool",
"cutoff_frequency": 0.001
}
}
}

I couldn't find any example of how to do this.

Can you please let me know how should i modify my query to include this ?

{
"timeout": 30000,
"highlight": {
"pre_tags": [
""
],
"post_tags": [
""
],
"fields": {
"srId": {
"number_of_fragments": 0
},
"emails.emailBody": {
"number_of_fragments": 0
},
"chatTextArray.text": {
"number_of_fragments": 0
}
}
},
"query": {
"filtered": {
"filter": [
{
"range": {
"closeDt": {
"from": 1325404800000,
"to": 1391760000000
}
}
}
],
"query": {
"query_string": {
"query": "payment button",
"default_operator": "AND",
"fields": [
[
"emails.emailBody",
"srId",
"chatTextArray.text"
]
]
}
},
"_cache": true
}
},
"size": 10
}

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/common-terms-query-with-cutoff-frequency-tp4049060.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/1391713109922-4049060.post%40n3.nabble.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGCwEM_c37QxQm7o73QKoKyxpzUWAtz_8s12czN-oArm9SF6Nw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

vinamar · February 11, 2014, 10:28pm

Hi Alex,

I used the multi_match query to query with stop words but the returned results doesn't contain the stop words. I used exact match with double quotes on the query string. Also the highlighted results - doesn't contain the stop words in the results.

"query": {
"multi_match": {
"query": ""not%20received"",
"operator": "AND",
"zero_terms_query": "all",
"fields": [
[
"emails.emailBody",
"srId",
"chatTextArray.text"
]
]
}
}

spinscale · February 12, 2014, 1:43pm

Hey,

you may have used an analyzer which removed stopwords? Try with another one
and see if that works... side note: The default analyzer of elasticsearch
0.90 removes stopwords, the default one of 1.x does not, so take care of
the version you are trying this iwth.

--Alex

On Tue, Feb 11, 2014 at 11:28 PM, vinamar vinamar@gmail.com wrote:

Hi Alex,

I used the multi_match query to query with stop words but the returned
results doesn't contain the stop words. I used exact match with double
quotes on the query string. Also the highlighted results - doesn't contain
the stop words in the results.

"query": {
"multi_match": {
"query": ""not%20received"",
"operator": "AND",
"zero_terms_query": "all",
"fields": [
[
"emails.emailBody",
"srId",
"chatTextArray.text"
]
]
}
}

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/common-terms-query-with-cutoff-frequency-tp4049060p4049410.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/1392157721335-4049410.post%40n3.nabble.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGCwEM8aomx_f_CkS%2BTK1W52wU%2BXPCHtPdWuzsHqKm5jNTtn4w%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

vinamar · February 12, 2014, 10:45pm

Hi Alex,

I tested on our sandbox with ES 1.0 with a new data set - search defaults to including stop words in highlighted results.

For this change to happen do we need to re-index data after switching to ES 1.0 ? or the default analyzer would not be used during query time ?

Thanks, Vinoth

spinscale · February 13, 2014, 9:05am

Hey,

you need to reindex into a new index. Your old indices keep using the
analyzer which has been defined earlier before the first data was indexed
(switching that in between would lead to pretty unpredictable results from
a search point of view).

--Alex

On Wed, Feb 12, 2014 at 11:45 PM, vinamar vinamar@gmail.com wrote:

Hi Alex,

I tested on our sandbox with ES 1.0 with a new data set - search
defaults
to including stop words in highlighted results.

For this change to happen do we need to re-index data after switching to ES
1.0 ? or the default analyzer would not be used during query time ?

Thanks, Vinoth

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/common-terms-query-with-cutoff-frequency-tp4049060p4049504.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/1392245145265-4049504.post%40n3.nabble.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGCwEM8FdrSutEcYMwiRum4QURNA6n0WE9D93fYcPUFcP%3DH6MA%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.