# Query analzyer with respect to field/index analzyer
**URL:** https://discuss.elastic.co/t/query-analzyer-with-respect-to-field-index-analzyer/13689
**Category:** Elasticsearch
**Created:** [September 19, 2013, 2:54pm UTC](https://discuss.elastic.co/t/query-analzyer-with-respect-to-field-index-analzyer/13689 "2013-09-19T14:54:58Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@Nora\_Olsen](https://discuss.elastic.co/u/Nora_Olsen)
#### Post date: [September 19, 2013, 2:54pm UTC](https://discuss.elastic.co/t/query-analzyer-with-respect-to-field-index-analzyer/13689/1 "2013-09-19T14:54:58Z")
Hi,
My mappings are defined like this:
{
"article" : {
"properties" : {
"text" : {"type" : "string", "store" : "yes",
"analyzer":"snowball"},
"country\_name" : {"type" : "string", "store" :
"yes", "analyzer":"snowball"},
"city\_name" : {"type" : "string", "store" :
"yes", "analyzer":"snowball"},
"tags": {"type":"string", "store":"no",
"analyzer":"snowball"},
}
}
}
And I have the following query:
"query" : {
"multi\_match" : {
"fields" : ["fieldA", "fieldB"],
"query" : query
}
},
I have read that the search query analyzer should match with the
index analyzer but what if one of the properties uses an EdgeNGram?
Also, if I specify "snowball" in my query, the results are flipped. Should
it be since both the query and index are using the same analyzer?
Thanks!
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@javanna](https://discuss.elastic.co/u/javanna)
#### Post date: [September 19, 2013, 4:17pm UTC](https://discuss.elastic.co/t/query-analzyer-with-respect-to-field-index-analzyer/13689/2 "2013-09-19T16:17:00Z")
Hi Nora,
correct usually the analyzer used at querytime is similar to the one used
at index time. NGrams are one of the exceptions, which usually get applied
only at index time as you have guessed.
If you want to use a different analyzer at querytime you can either specify
it in the query or in the mapping (search\_analyzer).
If you apply stemming, it's usually needed to apply it at both index time
and query time, as you only index the stems and that's what you want to
search for (only the query terms stems).
That said, specifying snowball in your query shouldn't make any difference
as you have the same in the mapping (analyzer means both index\_analyzer and
search\_analyzer will be the same). Do you have a different score for your
documents or always the same?
On Thursday, September 19, 2013 4:54:58 PM UTC+2, Nora Olsen wrote:
> Hi,
>
> My mappings are defined like this:
> {
> "article" : {
> "properties" : {
> "text" : {"type" : "string", "store" : "yes",
> "analyzer":"snowball"},
> "country\_name" : {"type" : "string", "store" :
> "yes", "analyzer":"snowball"},
> "city\_name" : {"type" : "string", "store" :
> "yes", "analyzer":"snowball"},
> "tags": {"type":"string", "store":"no",
> "analyzer":"snowball"},
> }
> }
> }
>
> And I have the following query:
> "query" : {
> "multi\_match" : {
> "fields" : ["fieldA", "fieldB"],
> "query" : query
> }
> },
>
> I have read that the search query analyzer should match with the
> index analyzer but what if one of the properties uses an EdgeNGram?
>
> Also, if I specify "snowball" in my query, the results are flipped. Should
> it be since both the query and index are using the same analyzer?
>
> Thanks!
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@Nora\_Olsen](https://discuss.elastic.co/u/Nora_Olsen)
#### Post date: [September 19, 2013, 4:55pm UTC](https://discuss.elastic.co/t/query-analzyer-with-respect-to-field-index-analzyer/13689/3 "2013-09-19T16:55:32Z")
Looks like is a mistake from me. The results are the same regardless
whether the 'snowball' analyzer is specified in the query.
Is it common to have different analyzers on various fields, except ngrams?
Based on my document:
"properties" : {
"text" : {"type" : "string", "store" : "yes",
"analyzer":"snowball"},
"country\_name" : {"type" : "string", "store" :
"yes", "analyzer":"snowball"},
"city\_name" : {"type" : "string", "store" :
"yes", "analyzer":"snowball"},
"tags": {"type":"string", "store":"no",
"analyzer":"snowball"},
}
And I'm trying to perform queries without the use NLP POS tagging or Entity
Extraction:
"multi\_match" : {
"analyzer": "snowball",
"fields" : ["text", "city\_name^1.3",
"country\_name^1.2", "tags^1.1"],
"query" : "Hiking trails in Brazil"
}
I am thinking "snowball" should be sufficient?
Thanks!
On Friday, September 20, 2013 12:17:00 AM UTC+8, Luca Cavanna wrote:
> Hi Nora,
> correct usually the analyzer used at querytime is similar to the one used
> at index time. NGrams are one of the exceptions, which usually get applied
> only at index time as you have guessed.
> If you want to use a different analyzer at querytime you can either
> specify it in the query or in the mapping (search\_analyzer).
>
> If you apply stemming, it's usually needed to apply it at both index time
> and query time, as you only index the stems and that's what you want to
> search for (only the query terms stems).
>
> That said, specifying snowball in your query shouldn't make any difference
> as you have the same in the mapping (analyzer means both index\_analyzer and
> search\_analyzer will be the same). Do you have a different score for your
> documents or always the same?
>
> On Thursday, September 19, 2013 4:54:58 PM UTC+2, Nora Olsen wrote:
>
> > Hi,
> >
> > My mappings are defined like this:
> > {
> > "article" : {
> > "properties" : {
> > "text" : {"type" : "string", "store" : "yes",
> > "analyzer":"snowball"},
> > "country\_name" : {"type" : "string", "store"
> > : "yes", "analyzer":"snowball"},
> > "city\_name" : {"type" : "string", "store" :
> > "yes", "analyzer":"snowball"},
> > "tags": {"type":"string", "store":"no",
> > "analyzer":"snowball"},
> > }
> > }
> > }
> >
> > And I have the following query:
> > "query" : {
> > "multi\_match" : {
> > "fields" : ["fieldA", "fieldB"],
> > "query" : query
> > }
> > },
> >
> > I have read that the search query analyzer should match with the
> > index analyzer but what if one of the properties uses an EdgeNGram?
> >
> > Also, if I specify "snowball" in my query, the results are flipped.
> > Should it be since both the query and index are using the same analyzer?
> >
> > Thanks!
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@javanna](https://discuss.elastic.co/u/javanna)
#### Post date: [September 19, 2013, 5:03pm UTC](https://discuss.elastic.co/t/query-analzyer-with-respect-to-field-index-analzyer/13689/4 "2013-09-19T17:03:46Z")
Yes it is common to have different analyzers per field. For instance, it
usually doesn't make sense to apply stemming on city or country names,
probably not even titles as you somehow lose information. I would apply
stemming on a long text. Same for stopwords. What's important is
lowercasing for instance, which is something that I've seen applied pretty
much on every field.
I would suggest to play around with analyzer and build your own text
analysis chain, per field, depending on your data and requirements. Take a
look at the analyze
api[http://www.elasticsearch.org/guide/reference/api/admin-indices-analyze/](http://www.elasticsearch.org/guide/reference/api/admin-indices-analyze/)and
the inquisitor
plugin [https://github.com/polyfractal/elasticsearch-inquisitor](https://github.com/polyfractal/elasticsearch-inquisitor) to see how
the whole thing works.
Cheers
Luca
On Thu, Sep 19, 2013 at 6:55 PM, Nora Olsen [nora.olsen77@gmail.com](mailto:nora.olsen77@gmail.com) wrote:
> Looks like is a mistake from me. The results are the same regardless
> whether the 'snowball' analyzer is specified in the query.
>
> Is it common to have different analyzers on various fields, except ngrams?
>
> Based on my document:
> "properties" : {
> "text" : {"type" : "string", "store" : "yes",
> "analyzer":"snowball"},
> "country\_name" : {"type" : "string", "store" :
> "yes", "analyzer":"snowball"},
> "city\_name" : {"type" : "string", "store" :
> "yes", "analyzer":"snowball"},
> "tags": {"type":"string", "store":"no",
> "analyzer":"snowball"},
> }
>
> And I'm trying to perform queries without the use NLP POS tagging or
> Entity Extraction:
> "multi\_match" : {
> "analyzer": "snowball",
> "fields" : ["text", "city\_name^1.3",
> "country\_name^1.2", "tags^1.1"],
> "query" : "Hiking trails in Brazil"
> }
>
> I am thinking "snowball" should be sufficient?
>
> Thanks!
>
> On Friday, September 20, 2013 12:17:00 AM UTC+8, Luca Cavanna wrote:
>
> > Hi Nora,
> > correct usually the analyzer used at querytime is similar to the one used
> > at index time. NGrams are one of the exceptions, which usually get applied
> > only at index time as you have guessed.
> > If you want to use a different analyzer at querytime you can either
> > specify it in the query or in the mapping (search\_analyzer).
> >
> > If you apply stemming, it's usually needed to apply it at both index time
> > and query time, as you only index the stems and that's what you want to
> > search for (only the query terms stems).
> >
> > That said, specifying snowball in your query shouldn't make any
> > difference as you have the same in the mapping (analyzer means both
> > index\_analyzer and search\_analyzer will be the same). Do you have a
> > different score for your documents or always the same?
> >
> > On Thursday, September 19, 2013 4:54:58 PM UTC+2, Nora Olsen wrote:
> >
> > > Hi,
> > >
> > > My mappings are defined like this:
> > > {
> > > "article" : {
> > > "properties" : {
> > > "text" : {"type" : "string", "store" :
> > > "yes", "analyzer":"snowball"},
> > > "country\_name" : {"type" : "string", "store"
> > > : "yes", "analyzer":"snowball"},
> > > "city\_name" : {"type" : "string", "store" :
> > > "yes", "analyzer":"snowball"},
> > > "tags": {"type":"string", "store":"no",
> > > "analyzer":"snowball"},
> > > }
> > > }
> > > }
> > >
> > > And I have the following query:
> > > "query" : {
> > > "multi\_match" : {
> > > "fields" : ["fieldA", "fieldB"],
> > > "query" : query
> > > }
> > > },
> > >
> > > I have read that the search query analyzer should match with the
> > > index analyzer but what if one of the properties uses an EdgeNGram?
> > >
> > > Also, if I specify "snowball" in my query, the results are flipped.
> > > Should it be since both the query and index are using the same analyzer?
> > >
> > > Thanks!
> > >
> > > --
> > > You received this message because you are subscribed to a topic in the
> > > Google Groups "elasticsearch" group.
> > > To unsubscribe from this topic, visit
> > > [https://groups.google.com/d/topic/elasticsearch/am5sX8-3w8U/unsubscribe](https://groups.google.com/d/topic/elasticsearch/am5sX8-3w8U/unsubscribe).
> > > To unsubscribe from this group and all its topics, send an email to
> > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@Nora\_Olsen](https://discuss.elastic.co/u/Nora_Olsen)
#### Post date: [September 19, 2013, 5:14pm UTC](https://discuss.elastic.co/t/query-analzyer-with-respect-to-field-index-analzyer/13689/5 "2013-09-19T17:14:09Z")
Thanks for the pointers. Will read up more.
On Friday, September 20, 2013 1:03:46 AM UTC+8, Luca Cavanna wrote:
> Yes it is common to have different analyzers per field. For instance, it
> usually doesn't make sense to apply stemming on city or country names,
> probably not even titles as you somehow lose information. I would apply
> stemming on a long text. Same for stopwords. What's important is
> lowercasing for instance, which is something that I've seen applied pretty
> much on every field.
>
> I would suggest to play around with analyzer and build your own text
> analysis chain, per field, depending on your data and requirements. Take a
> look at the analyze api[http://www.elasticsearch.org/guide/reference/api/admin-indices-analyze/](http://www.elasticsearch.org/guide/reference/api/admin-indices-analyze/)and the inquisitor
> plugin [https://github.com/polyfractal/elasticsearch-inquisitor](https://github.com/polyfractal/elasticsearch-inquisitor) to see
> how the whole thing works.
>
> Cheers
> Luca
>
> On Thu, Sep 19, 2013 at 6:55 PM, Nora Olsen \<[nora.o...@gmail.com](mailto:nora.o...@gmail.com)\
>
> > wrote:
>
> > Looks like is a mistake from me. The results are the same regardless
> > whether the 'snowball' analyzer is specified in the query.
> >
> > Is it common to have different analyzers on various fields, except
> > ngrams?
> >
> > Based on my document:
> > "properties" : {
> > "text" : {"type" : "string", "store" : "yes",
> > "analyzer":"snowball"},
> > "country\_name" : {"type" : "string", "store"
> > : "yes", "analyzer":"snowball"},
> > "city\_name" : {"type" : "string", "store" :
> > "yes", "analyzer":"snowball"},
> > "tags": {"type":"string", "store":"no",
> > "analyzer":"snowball"},
> > }
> >
> > And I'm trying to perform queries without the use NLP POS tagging or
> > Entity Extraction:
> > "multi\_match" : {
> > "analyzer": "snowball",
> > "fields" : ["text", "city\_name^1.3",
> > "country\_name^1.2", "tags^1.1"],
> > "query" : "Hiking trails in Brazil"
> > }
> >
> > I am thinking "snowball" should be sufficient?
> >
> > Thanks!
> >
> > On Friday, September 20, 2013 12:17:00 AM UTC+8, Luca Cavanna wrote:
> >
> > > Hi Nora,
> > > correct usually the analyzer used at querytime is similar to the one
> > > used at index time. NGrams are one of the exceptions, which usually get
> > > applied only at index time as you have guessed.
> > > If you want to use a different analyzer at querytime you can either
> > > specify it in the query or in the mapping (search\_analyzer).
> > >
> > > If you apply stemming, it's usually needed to apply it at both index
> > > time and query time, as you only index the stems and that's what you want
> > > to search for (only the query terms stems).
> > >
> > > That said, specifying snowball in your query shouldn't make any
> > > difference as you have the same in the mapping (analyzer means both
> > > index\_analyzer and search\_analyzer will be the same). Do you have a
> > > different score for your documents or always the same?
> > >
> > > On Thursday, September 19, 2013 4:54:58 PM UTC+2, Nora Olsen wrote:
> > >
> > > > Hi,
> > > >
> > > > My mappings are defined like this:
> > > > {
> > > > "article" : {
> > > > "properties" : {
> > > > "text" : {"type" : "string", "store" :
> > > > "yes", "analyzer":"snowball"},
> > > > "country\_name" : {"type" : "string",
> > > > "store" : "yes", "analyzer":"snowball"},
> > > > "city\_name" : {"type" : "string", "store" :
> > > > "yes", "analyzer":"snowball"},
> > > > "tags": {"type":"string", "store":"no",
> > > > "analyzer":"snowball"},
> > > > }
> > > > }
> > > > }
> > > >
> > > > And I have the following query:
> > > > "query" : {
> > > > "multi\_match" : {
> > > > "fields" : ["fieldA", "fieldB"],
> > > > "query" : query
> > > > }
> > > > },
> > > >
> > > > I have read that the search query analyzer should match with the
> > > > index analyzer but what if one of the properties uses an EdgeNGram?
> > > >
> > > > Also, if I specify "snowball" in my query, the results are flipped.
> > > > Should it be since both the query and index are using the same analyzer?
> > > >
> > > > Thanks!
> > > >
> > > > --
> > > > You received this message because you are subscribed to a topic in the
> > > > Google Groups "elasticsearch" group.
> > > > To unsubscribe from this topic, visit
> > > > [https://groups.google.com/d/topic/elasticsearch/am5sX8-3w8U/unsubscribe](https://groups.google.com/d/topic/elasticsearch/am5sX8-3w8U/unsubscribe).
> > > > To unsubscribe from this group and all its topics, send an email to
> > > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \.
> > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 2:15am UTC](https://discuss.elastic.co/t/query-analzyer-with-respect-to-field-index-analzyer/13689/6 "2017-07-06T02:15:28Z")