# How does shingle filter work on match\_phrase in query phase? **URL:** \ **Category:** Elasticsearch\ **Created:** [June 20, 2014, 5:04am UTC](https://discuss.elastic.co/t/how-does-shingle-filter-work-on-match-phrase-in-query-phase/18225 "2014-06-20T05:04:06Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![11153](https://avatars.discourse-cdn.com/v4/letter/1/e274bd/32.png) [@11153](https://discuss.elastic.co/u/11153)\ **Post date:** [June 20, 2014, 5:04am UTC](https://discuss.elastic.co/t/how-does-shingle-filter-work-on-match-phrase-in-query-phase/18225/1 "2014-06-20T05:04:06Z")
How does shingle filter work on match\_phrase in query phase? After analyzing phrase "t1 t2 t3", shingle filter produced five tokens, t1 t2 t3 "t1 t2" "t2 t3" Will match\_phrase still give "t1 t2 t3" a match? How it works? Thank you. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/33889bbd-9b01-4414-b579-4e625f0eec17%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/33889bbd-9b01-4414-b579-4e625f0eec17%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Cedric\_Hourcade](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cedric_hourcade/32/1451_2.png) [@Cedric\_Hourcade](https://discuss.elastic.co/u/Cedric_Hourcade)\ **Post date:** [June 20, 2014, 8:17am UTC](https://discuss.elastic.co/t/how-does-shingle-filter-work-on-match-phrase-in-query-phase/18225/2 "2014-06-20T08:17:34Z")
Hello, Let's say you have an indexed text "t1 t3 t3" with shingles. The token positions are also indexed, so you get : t1 (at pos 1), "t1 t2" (pos 1), t2 (pos 2), "t2 t3" (pos 2) and t3 (pos 3). So if you are searching with a match\_phrase for "t1 t2 t3" (even if not tokenized as shingles) it will matches the document, because t1, t2 and t3 are considered next to each others (based on there recorded position) for this document. Cédric Hourcade [ced@wal.fr](mailto:ced@wal.fr) On Fri, Jun 20, 2014 at 7:04 AM, 陳智清 [walker0902@gmail.com](mailto:walker0902@gmail.com) wrote: > How does shingle filter work on match\_phrase in query phase? > > After analyzing phrase "t1 t2 t3", shingle filter produced five tokens, > t1 > t2 > t3 > "t1 t2" > "t2 t3" > > Will match\_phrase still give "t1 t2 t3" a match? How it works? Thank you. > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/33889bbd-9b01-4414-b579-4e625f0eec17%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/33889bbd-9b01-4414-b579-4e625f0eec17%40googlegroups.com). > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAJQxjPNWyj-r6LtrWDXv\_HGA-sgxfy%3DEu4Z5gJ5kRk\_K2MWVNw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAJQxjPNWyj-r6LtrWDXv_HGA-sgxfy%3DEu4Z5gJ5kRk_K2MWVNw%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![11153](https://avatars.discourse-cdn.com/v4/letter/1/e274bd/32.png) [@11153](https://discuss.elastic.co/u/11153)\ **Post date:** [June 20, 2014, 10:30am UTC](https://discuss.elastic.co/t/how-does-shingle-filter-work-on-match-phrase-in-query-phase/18225/3 "2014-06-20T10:30:52Z")
Hello Hourcade, Thanks for your response. Does that mean different values should be set to "index\_analyzer" and "search\_analyzer"? (e.g. "index\_analyzer": "shingle", and "search\_analyzer": "standard") What if I want to re-use the same "shingle" analyzer in both index and search? will the match\_phrase "t1 t2 t3" still give me a match? I know that set a different analyzer to "search\_analyzer" makes match\_phrase "t1 t2 t3" searchable, but if I do that, then I get no benefit from "shingle", right? Instead I get a bigger index size. I assume "shingle" is used for faster "match\_phrase" searches. But after shingle, searching a phrase of 3 tokens "t1 t2 t3" becomes searching a phrase of 5 tokens plus I don't know how "shingle" arranges the positions for a correct phrase query. So how can "match\_phrase" be faster? Thank you. Cédric Hourcade於 2014年6月20日星期五UTC+8下午4時18分03秒寫道： > Hello, > > Let's say you have an indexed text "t1 t3 t3" with shingles. The token > positions are also indexed, so you get : t1 (at pos 1), "t1 t2" (pos > 1), t2 (pos 2), "t2 t3" (pos 2) and t3 (pos 3). > > So if you are searching with a match\_phrase for "t1 t2 t3" (even if > not tokenized as shingles) it will matches the document, because t1, > t2 and t3 are considered next to each others (based on there recorded > position) for this document. > > Cédric Hourcade > [c...@wal.fr](mailto:c...@wal.fr) \ > > On Fri, Jun 20, 2014 at 7:04 AM, 陳智清 \<[walke...@gmail.com](mailto:walke...@gmail.com) \\> > wrote: > > > How does shingle filter work on match\_phrase in query phase? > > > > After analyzing phrase "t1 t2 t3", shingle filter produced five tokens, > > t1 > > t2 > > t3 > > "t1 t2" > > "t2 t3" > > > > Will match\_phrase still give "t1 t2 t3" a match? How it works? Thank > > you. > > > > -- > > You received this message because you are subscribed to the Google > > Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send > > an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/33889bbd-9b01-4414-b579-4e625f0eec17%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/33889bbd-9b01-4414-b579-4e625f0eec17%40googlegroups.com). > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/602477cb-d8f4-459b-8888-e6174662fbfd%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/602477cb-d8f4-459b-8888-e6174662fbfd%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Cedric\_Hourcade](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cedric_hourcade/32/1451_2.png) [@Cedric\_Hourcade](https://discuss.elastic.co/u/Cedric_Hourcade)\ **Post date:** [June 20, 2014, 12:00pm UTC](https://discuss.elastic.co/t/how-does-shingle-filter-work-on-match-phrase-in-query-phase/18225/4 "2014-06-20T12:00:12Z")
Yes, you can use two different analyzers. In your case what you can do is: - for the the indexation you apply a shingle filter. - for the query you also apply a shingle filter, but this time you disable the unigrams (output\_unigrams: false), so it will only generate the shingles, in your case : "t1 t2" and "t2 t3". It will match your document. Cédric Hourcade [ced@wal.fr](mailto:ced@wal.fr) On Fri, Jun 20, 2014 at 12:30 PM, 陳智清 [walker0902@gmail.com](mailto:walker0902@gmail.com) wrote: > Hello Hourcade, Thanks for your response. > > Does that mean different values should be set to "index\_analyzer" and > "search\_analyzer"? (e.g. "index\_analyzer": "shingle", and "search\_analyzer": > "standard") > What if I want to re-use the same "shingle" analyzer in both index and > search? will the match\_phrase "t1 t2 t3" still give me a match? > > I know that set a different analyzer to "search\_analyzer" makes match\_phrase > "t1 t2 t3" searchable, but if I do that, then I get no benefit from > "shingle", right? Instead I get a bigger index size. > > I assume "shingle" is used for faster "match\_phrase" searches. But after > shingle, searching a phrase of 3 tokens "t1 t2 t3" becomes searching a > phrase of 5 tokens plus I don't know how "shingle" arranges the positions > for a correct phrase query. So how can "match\_phrase" be faster? Thank you. > > Cédric Hourcade於 2014年6月20日星期五UTC+8下午4時18分03秒寫道： > > > Hello, > > > > Let's say you have an indexed text "t1 t3 t3" with shingles. The token > > positions are also indexed, so you get : t1 (at pos 1), "t1 t2" (pos > > 1), t2 (pos 2), "t2 t3" (pos 2) and t3 (pos 3). > > > > So if you are searching with a match\_phrase for "t1 t2 t3" (even if > > not tokenized as shingles) it will matches the document, because t1, > > t2 and t3 are considered next to each others (based on there recorded > > position) for this document. > > > > Cédric Hourcade > > [c...@wal.fr](mailto:c...@wal.fr) > > > > On Fri, Jun 20, 2014 at 7:04 AM, 陳智清 [walke...@gmail.com](mailto:walke...@gmail.com) wrote: > > > > > How does shingle filter work on match\_phrase in query phase? > > > > > > After analyzing phrase "t1 t2 t3", shingle filter produced five tokens, > > > t1 > > > t2 > > > t3 > > > "t1 t2" > > > "t2 t3" > > > > > > Will match\_phrase still give "t1 t2 t3" a match? How it works? Thank > > > you. > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups > > > "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an > > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > To view this discussion on the web visit > > > > > > [https://groups.google.com/d/msgid/elasticsearch/33889bbd-9b01-4414-b579-4e625f0eec17%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/33889bbd-9b01-4414-b579-4e625f0eec17%40googlegroups.com). > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/602477cb-d8f4-459b-8888-e6174662fbfd%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/602477cb-d8f4-459b-8888-e6174662fbfd%40googlegroups.com). > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAJQxjPMAEGK%3DSxYfoBtjgcdZYPHqAAiSPpQBjh1fvtXgkwWuLA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAJQxjPMAEGK%3DSxYfoBtjgcdZYPHqAAiSPpQBjh1fvtXgkwWuLA%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![11153](https://avatars.discourse-cdn.com/v4/letter/1/e274bd/32.png) [@11153](https://discuss.elastic.co/u/11153)\ **Post date:** [June 20, 2014, 1:11pm UTC](https://discuss.elastic.co/t/how-does-shingle-filter-work-on-match-phrase-in-query-phase/18225/5 "2014-06-20T13:11:42Z")
I got it! Thank you!!!!! Cédric Hourcade於 2014年6月20日星期五UTC+8下午8時00分36秒寫道： > Yes, you can use two different analyzers. In your case what you can do is: > > - for the the indexation you apply a shingle filter. > - for the query you also apply a shingle filter, but this time you > disable the unigrams (output\_unigrams: false), so it will only > generate the shingles, in your case : "t1 t2" and "t2 t3". It will > match your document. > Cédric Hourcade > [c...@wal.fr](mailto:c...@wal.fr) \ > > On Fri, Jun 20, 2014 at 12:30 PM, 陳智清 \<[walke...@gmail.com](mailto:walke...@gmail.com) \\> > wrote: > > > Hello Hourcade, Thanks for your response. > > > > Does that mean different values should be set to "index\_analyzer" and > > "search\_analyzer"? (e.g. "index\_analyzer": "shingle", and > > "search\_analyzer": > > "standard") > > What if I want to re-use the same "shingle" analyzer in both index and > > search? will the match\_phrase "t1 t2 t3" still give me a match? > > > > I know that set a different analyzer to "search\_analyzer" makes > > match\_phrase > > "t1 t2 t3" searchable, but if I do that, then I get no benefit from > > "shingle", right? Instead I get a bigger index size. > > > > I assume "shingle" is used for faster "match\_phrase" searches. But after > > shingle, searching a phrase of 3 tokens "t1 t2 t3" becomes searching a > > phrase of 5 tokens plus I don't know how "shingle" arranges the > > positions > > for a correct phrase query. So how can "match\_phrase" be faster? Thank > > you. > > > > Cédric Hourcade於 2014年6月20日星期五UTC+8下午4時18分03秒寫道： > > > > > Hello, > > > > > > Let's say you have an indexed text "t1 t3 t3" with shingles. The token > > > positions are also indexed, so you get : t1 (at pos 1), "t1 t2" (pos > > > 1), t2 (pos 2), "t2 t3" (pos 2) and t3 (pos 3). > > > > > > So if you are searching with a match\_phrase for "t1 t2 t3" (even if > > > not tokenized as shingles) it will matches the document, because t1, > > > t2 and t3 are considered next to each others (based on there recorded > > > position) for this document. > > > > > > Cédric Hourcade > > > [c...@wal.fr](mailto:c...@wal.fr) > > > > > > On Fri, Jun 20, 2014 at 7:04 AM, 陳智清 [walke...@gmail.com](mailto:walke...@gmail.com) wrote: > > > > > > > How does shingle filter work on match\_phrase in query phase? > > > > > > > > After analyzing phrase "t1 t2 t3", shingle filter produced five > > > > tokens, > > > > t1 > > > > t2 > > > > t3 > > > > "t1 t2" > > > > "t2 t3" > > > > > > > > Will match\_phrase still give "t1 t2 t3" a match? How it works? Thank > > > > you. > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups > > > > "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, > > > > send > > > > an > > > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/33889bbd-9b01-4414-b579-4e625f0eec17%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/33889bbd-9b01-4414-b579-4e625f0eec17%40googlegroups.com). > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to the Google > > Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send > > an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/602477cb-d8f4-459b-8888-e6174662fbfd%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/602477cb-d8f4-459b-8888-e6174662fbfd%40googlegroups.com). > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/61083ccb-f678-4074-bd48-a4dbcc0c0511%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/61083ccb-f678-4074-bd48-a4dbcc0c0511%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:20am UTC](https://discuss.elastic.co/t/how-does-shingle-filter-work-on-match-phrase-in-query-phase/18225/6 "2017-07-06T01:20:56Z")