# Confused about Shingle behavior **URL:** \ **Category:** Elasticsearch\ **Created:** [November 13, 2012, 4:56am UTC](https://discuss.elastic.co/t/confused-about-shingle-behavior/9689 "2012-11-13T04:56:52Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![polyfractal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/polyfractal/32/48162_2.png) [@polyfractal](https://discuss.elastic.co/u/polyfractal)\ **Post date:** [November 13, 2012, 4:56am UTC](https://discuss.elastic.co/t/confused-about-shingle-behavior/9689/1 "2012-11-13T04:56:52Z")
I'm really struggling to get proper shingle searching to work. I've tried dozens of variations, using text, string\_query, bools, dis\_max. The whole works. I simply cannot get it to function the way that I want. I imagine I'm doing something fundamentally wrong, since this seems like an easy behavior. My mapping looks like this: [https://gist.github.com/4063964](https://gist.github.com/4063964) Basically, I'm indexing a field with a normal tokenizer as well as a shingle tokenizer. With regards to search, I want to match exact phrases first, then match shingled phrases next (e.g. partial phrases). I'm searching for "Great Planes Rotor Blade" using the following query: { "explain":true, "size":5, "from":0, "highlight":{ "pre\_tags":[ "**" ], "post\_tags":[ "**" ], "fields":{ "body":{ ``` } } ``` }, "query":{ "dis\_max":{ "tie\_breaker":0.7, "queries":[ { "text":{ "body":{ "query":"Great Planes Rotor Blade", "type":"phrase" } } }, { "query\_string":{ "fields":[ "body" ], "query":"Great Planes Rotor Blade", "phrase\_slop":0, "minimum\_should\_match":"40%" } }, { "query\_string":{ "fields":[ "body" ], "query":"Great Planes Rotor Blade", "phrase\_slop":0, "minimum\_should\_match":"40%", "analyzer":"analyzer\_partial\_shingle" } } ] } } } Unfortunately, I'm getting results all over the place. Some items which use the word "blade" 4-5 times will rank higher than items that use the phrase "Great Planes" once. I assumed that shingling the query (using analyzer\_partial\_shingle) and then searching the indexed shingles would find "Great Planes" and increase the score, but it doesn't seem to be working that way. Anyone shed some light on what I'm doing wrong? -- ---
**Author:** ![polyfractal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/polyfractal/32/48162_2.png) [@polyfractal](https://discuss.elastic.co/u/polyfractal)\ **Post date:** [November 13, 2012, 1:42pm UTC](https://discuss.elastic.co/t/confused-about-shingle-behavior/9689/2 "2012-11-13T13:42:26Z")
As a followup, I've been toying with the bare minimum required to get shingles working. This is my current query iteration, but it doesn't return any results and I'm unsure why: { "explain":true, "size":5, "from":0, "highlight":{ "pre\_tags":[ "**\" ], "post\_tags":[ "**" ], "fields":{ "body":{ ``` } } ``` }, "query":{ "text\_phrase":{ "body.partial\_shingle":{ "query":"Great Planes Rotor Blade", "analyzer":"analyzer\_partial\_shingle" } } } } If I'm understanding correctly, this text\_phrase query should 1. Break the query "Great Planes Rotor Blade" into two bi-grams using \* analyzer\_partial\_shingle\*: ["Great Planes", "Planes Rotor"]. I've set unigrams = false, so only these two bigrams should be produced 2. Text\_phrase will take these two tokens and and perform an exact phrase match against body.partial\_shingle. 3. Any document with "Great Planes" as a token should be found. Clearly I'm missing something, since this is not happening. Any tips? Thanks! -Zach On Monday, November 12, 2012 11:56:52 PM UTC-5, Zachary Tong wrote: > I'm really struggling to get proper shingle searching to work. I've tried > dozens of variations, using text, string\_query, bools, dis\_max. The whole > works. I simply cannot get it to function the way that I want. I imagine > I'm doing something fundamentally wrong, since this seems like an easy > behavior. My mapping looks like this: [ES Mapping · GitHub](https://gist.github.com/4063964) > > Basically, I'm indexing a field with a normal tokenizer as well as a > shingle tokenizer. With regards to search, I want to match exact phrases > first, then match shingled phrases next (e.g. partial phrases). I'm > searching for "Great Planes Rotor Blade" using the following query: > > { > "explain":true, > "size":5, > "from":0, > "highlight":{ > "pre\_tags":[ > "**" > ], > "post\_tags":[ > "**" > ], > "fields":{ > "body":{ > > ``` > } > } > > ``` > > }, > "query":{ > "dis\_max":{ > "tie\_breaker":0.7, > "queries":[ > { > "text":{ > "body":{ > "query":"Great Planes Rotor Blade", > "type":"phrase" > } > } > }, > { > "query\_string":{ > "fields":[ > "body" > ], > "query":"Great Planes Rotor Blade", > "phrase\_slop":0, > "minimum\_should\_match":"40%" > } > }, > { > "query\_string":{ > "fields":[ > "body" > ], > "query":"Great Planes Rotor Blade", > "phrase\_slop":0, > "minimum\_should\_match":"40%", > "analyzer":"analyzer\_partial\_shingle" > } > } > ] > } > } > } > > Unfortunately, I'm getting results all over the place. Some items which > use the word "blade" 4-5 times will rank higher than items that use the > phrase "Great Planes" once. I assumed that shingling the query (using > analyzer\_partial\_shingle) and then searching the indexed shingles would > find "Great Planes" and increase the score, but it doesn't seem to be > working that way. > > Anyone shed some light on what I'm doing wrong? -- ---
**Author:** ![Igor\_Motov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_motov/32/45193_2.png) [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)\ **Post date:** [November 13, 2012, 2:11pm UTC](https://discuss.elastic.co/t/confused-about-shingle-behavior/9689/3 "2012-11-13T14:11:17Z")
A couple of notes. 1) You are using a analyzer\_term as a search analyzer for body.partial\_shingle, so bi-grams don't actually happen on the query side. 2) not completely sure about your use case, but since you are using shingles, wouldn't it make more sense to just use text query instead of text\_phrase since shingles are handling the "phrase" aspect already. On Tuesday, November 13, 2012 8:42:27 AM UTC-5, Zachary Tong wrote: > As a followup, I've been toying with the bare minimum required to get > shingles working. This is my current query iteration, but it doesn't > return any results and I'm unsure why: > > { > "explain":true, > "size":5, > "from":0, > "highlight":{ > "pre\_tags":[ > "**\" > ], > "post\_tags":[ > "**" > ], > "fields":{ > "body":{ > > ``` > } > } > > ``` > > }, > "query":{ > "text\_phrase":{ > "body.partial\_shingle":{ > "query":"Great Planes Rotor Blade", > "analyzer":"analyzer\_partial\_shingle" > } > } > } > } > > If I'm understanding correctly, this text\_phrase query should > > 1. Break the query "Great Planes Rotor Blade" into two bi-grams using \* > analyzer\_partial\_shingle\*: ["Great Planes", "Planes Rotor"]. I've set > unigrams = false, so only these two bigrams should be produced > 2. Text\_phrase will take these two tokens and and perform an exact > phrase match against body.partial\_shingle. > 3. Any document with "Great Planes" as a token should be found. > > Clearly I'm missing something, since this is not happening. Any tips? > > Thanks! > -Zach > > On Monday, November 12, 2012 11:56:52 PM UTC-5, Zachary Tong wrote: > > > I'm really struggling to get proper shingle searching to work. I've > > tried dozens of variations, using text, string\_query, bools, dis\_max. The > > whole works. I simply cannot get it to function the way that I want. I > > imagine I'm doing something fundamentally wrong, since this seems like an > > easy behavior. My mapping looks like this: > > [ES Mapping · GitHub](https://gist.github.com/4063964) > > > > Basically, I'm indexing a field with a normal tokenizer as well as a > > shingle tokenizer. With regards to search, I want to match exact phrases > > first, then match shingled phrases next (e.g. partial phrases). I'm > > searching for "Great Planes Rotor Blade" using the following query: > > > > { > > "explain":true, > > "size":5, > > "from":0, > > "highlight":{ > > "pre\_tags":[ > > "**" > > ], > > "post\_tags":[ > > "**" > > ], > > "fields":{ > > "body":{ > > > > ``` > > } > > } > > > > ``` > > > > }, > > "query":{ > > "dis\_max":{ > > "tie\_breaker":0.7, > > "queries":[ > > { > > "text":{ > > "body":{ > > "query":"Great Planes Rotor Blade", > > "type":"phrase" > > } > > } > > }, > > { > > "query\_string":{ > > "fields":[ > > "body" > > ], > > "query":"Great Planes Rotor Blade", > > "phrase\_slop":0, > > "minimum\_should\_match":"40%" > > } > > }, > > { > > "query\_string":{ > > "fields":[ > > "body" > > ], > > "query":"Great Planes Rotor Blade", > > "phrase\_slop":0, > > "minimum\_should\_match":"40%", > > "analyzer":"analyzer\_partial\_shingle" > > } > > } > > ] > > } > > } > > } > > > > Unfortunately, I'm getting results all over the place. Some items which > > use the word "blade" 4-5 times will rank higher than items that use the > > phrase "Great Planes" once. I assumed that shingling the query (using > > analyzer\_partial\_shingle) and then searching the indexed shingles would > > find "Great Planes" and increase the score, but it doesn't seem to be > > working that way. > > > > Anyone shed some light on what I'm doing wrong? -- ---
**Author:** ![polyfractal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/polyfractal/32/48162_2.png) [@polyfractal](https://discuss.elastic.co/u/polyfractal)\ **Post date:** [November 13, 2012, 2:14pm UTC](https://discuss.elastic.co/t/confused-about-shingle-behavior/9689/4 "2012-11-13T14:14:25Z")
Does specifying the analyzer in the query not affect which search analyzer is used? If it doesn't, that's probably my problem! I'll try the text query instead. To be honest, I'm kinda flailing around. I'm not entirely certain how different queries work, so it's a lot of trial and error =) On Tuesday, November 13, 2012 9:11:17 AM UTC-5, Igor Motov wrote: > A couple of notes. 1) You are using a analyzer\_term as a search analyzer > for body.partial\_shingle, so bi-grams don't actually happen on the query > side. 2) not completely sure about your use case, but since you are using > shingles, wouldn't it make more sense to just use text query instead of > text\_phrase since shingles are handling the "phrase" aspect already. > > On Tuesday, November 13, 2012 8:42:27 AM UTC-5, Zachary Tong wrote: > > > As a followup, I've been toying with the bare minimum required to get > > shingles working. This is my current query iteration, but it doesn't > > return any results and I'm unsure why: > > > > { > > "explain":true, > > "size":5, > > "from":0, > > "highlight":{ > > "pre\_tags":[ > > "**\" > > ], > > "post\_tags":[ > > "**" > > ], > > "fields":{ > > "body":{ > > > > ``` > > } > > } > > > > ``` > > > > }, > > "query":{ > > "text\_phrase":{ > > "body.partial\_shingle":{ > > "query":"Great Planes Rotor Blade", > > "analyzer":"analyzer\_partial\_shingle" > > } > > } > > } > > } > > > > If I'm understanding correctly, this text\_phrase query should > > > > 1. Break the query "Great Planes Rotor Blade" into two bi-grams using > > _analyzer\_partial\_shingle_: ["Great Planes", "Planes Rotor"]. I've > > set unigrams = false, so only these two bigrams should be produced > > 2. Text\_phrase will take these two tokens and and perform an exact > > phrase match against body.partial\_shingle. > > 3. Any document with "Great Planes" as a token should be found. > > > > Clearly I'm missing something, since this is not happening. Any tips? > > > > Thanks! > > -Zach > > > > On Monday, November 12, 2012 11:56:52 PM UTC-5, Zachary Tong wrote: > > > > > I'm really struggling to get proper shingle searching to work. I've > > > tried dozens of variations, using text, string\_query, bools, dis\_max. The > > > whole works. I simply cannot get it to function the way that I want. I > > > imagine I'm doing something fundamentally wrong, since this seems like an > > > easy behavior. My mapping looks like this: > > > [ES Mapping · GitHub](https://gist.github.com/4063964) > > > > > > Basically, I'm indexing a field with a normal tokenizer as well as a > > > shingle tokenizer. With regards to search, I want to match exact phrases > > > first, then match shingled phrases next (e.g. partial phrases). I'm > > > searching for "Great Planes Rotor Blade" using the following query: > > > > > > { > > > "explain":true, > > > "size":5, > > > "from":0, > > > "highlight":{ > > > "pre\_tags":[ > > > "**" > > > ], > > > "post\_tags":[ > > > "**" > > > ], > > > "fields":{ > > > "body":{ > > > > > > ``` > > > } > > > } > > > > > > ``` > > > > > > }, > > > "query":{ > > > "dis\_max":{ > > > "tie\_breaker":0.7, > > > "queries":[ > > > { > > > "text":{ > > > "body":{ > > > "query":"Great Planes Rotor Blade", > > > "type":"phrase" > > > } > > > } > > > }, > > > { > > > "query\_string":{ > > > "fields":[ > > > "body" > > > ], > > > "query":"Great Planes Rotor Blade", > > > "phrase\_slop":0, > > > "minimum\_should\_match":"40%" > > > } > > > }, > > > { > > > "query\_string":{ > > > "fields":[ > > > "body" > > > ], > > > "query":"Great Planes Rotor Blade", > > > "phrase\_slop":0, > > > "minimum\_should\_match":"40%", > > > "analyzer":"analyzer\_partial\_shingle" > > > } > > > } > > > ] > > > } > > > } > > > } > > > > > > Unfortunately, I'm getting results all over the place. Some items which > > > use the word "blade" 4-5 times will rank higher than items that use the > > > phrase "Great Planes" once. I assumed that shingling the query (using > > > analyzer\_partial\_shingle) and then searching the indexed shingles would > > > find "Great Planes" and increase the score, but it doesn't seem to be > > > working that way. > > > > > > Anyone shed some light on what I'm doing wrong? -- ---
**Author:** ![Igor\_Motov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_motov/32/45193_2.png) [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)\ **Post date:** [November 13, 2012, 2:27pm UTC](https://discuss.elastic.co/t/confused-about-shingle-behavior/9689/5 "2012-11-13T14:27:32Z")
You are right, the analyzer specified on the query should be applied. I just didn't notice it: $ curl "localhost:9200/shingles/\_validate/query?pretty=true&explain=true" -d '{ "text":{ "body.partial\_shingle":{ "query":"Great Planes Rotor Blade", "analyzer":"analyzer\_partial\_shingle" } } }' { "valid" : true, "\_shards" : { "total" : 1, "successful" : 1, "failed" : 0 }, "explanations" : [ { "index" : "shingles", "valid" : true, "explanation" : "body.partial\_shingle:great planes body.partial\_shingle:great planes rotor body.partial\_shingle:great planes rotor blade body.partial\_shingle:planes rotor body.partial\_shingle:planes rotor blade body.partial\_shingle:rotor blade" } ] } By the way, the query\_string wouldn't work in your case because it splits terms by spaces in query parsing phase, before analyzer can actual get to the terms. On Tue, Nov 13, 2012 at 9:14 AM, Zachary Tong [zacharyjtong@gmail.com](mailto:zacharyjtong@gmail.com)wrote: > Does specifying the analyzer in the query not affect which search analyzer > is used? If it doesn't, that's probably my problem! > > I'll try the text query instead. To be honest, I'm kinda flailing around. > I'm not entirely certain how different queries work, so it's a lot of > trial and error =) > > On Tuesday, November 13, 2012 9:11:17 AM UTC-5, Igor Motov wrote: > > > A couple of notes. 1) You are using a analyzer\_term as a search analyzer > > for body.partial\_shingle, so bi-grams don't actually happen on the query > > side. 2) not completely sure about your use case, but since you are using > > shingles, wouldn't it make more sense to just use text query instead of > > text\_phrase since shingles are handling the "phrase" aspect already. > > > > On Tuesday, November 13, 2012 8:42:27 AM UTC-5, Zachary Tong wrote: > > > > > As a followup, I've been toying with the bare minimum required to get > > > shingles working. This is my current query iteration, but it doesn't > > > return any results and I'm unsure why: > > > > > > { > > > "explain":true, > > > "size":5, > > > "from":0, > > > "highlight":{ > > > "pre\_tags":[ > > > "**\" > > > ], > > > "post\_tags":[ > > > "**" > > > ], > > > "fields":{ > > > "body":{ > > > > > > ``` > > > } > > > } > > > > > > ``` > > > > > > }, > > > "query":{ > > > "text\_phrase":{ > > > "body.partial\_shingle":{ > > > "query":"Great Planes Rotor Blade", > > > "analyzer":"analyzer\_partial\_\*\*shingle" > > > } > > > } > > > } > > > } > > > > > > If I'm understanding correctly, this text\_phrase query should > > > > > > 1. Break the query "Great Planes Rotor Blade" into two bi-grams > > > using _analyzer\_partial\_shingle_: ["Great Planes", "Planes Rotor"]. > > > I've set unigrams = false, so only these two bigrams should be produced > > > 2. Text\_phrase will take these two tokens and and perform an exact > > > phrase match against body.partial\_shingle. > > > 3. Any document with "Great Planes" as a token should be found. > > > > > > Clearly I'm missing something, since this is not happening. Any tips? > > > > > > Thanks! > > > -Zach > > > > > > On Monday, November 12, 2012 11:56:52 PM UTC-5, Zachary Tong wrote: > > > > > > > I'm really struggling to get proper shingle searching to work. I've > > > > tried dozens of variations, using text, string\_query, bools, dis\_max. The > > > > whole works. I simply cannot get it to function the way that I want. I > > > > imagine I'm doing something fundamentally wrong, since this seems like an > > > > easy behavior. My mapping looks like this: [https://gist.github.com/](https://gist.github.com/)\*\* > > > > 4063964 [https://gist.github.com/4063964](https://gist.github.com/4063964) > > > > > > > > Basically, I'm indexing a field with a normal tokenizer as well as a > > > > shingle tokenizer. With regards to search, I want to match exact phrases > > > > first, then match shingled phrases next (e.g. partial phrases). I'm > > > > searching for "Great Planes Rotor Blade" using the following query: > > > > > > > > { > > > > "explain":true, > > > > "size":5, > > > > "from":0, > > > > "highlight":{ > > > > "pre\_tags":[ > > > > "**" > > > > ], > > > > "post\_tags":[ > > > > "**" > > > > ], > > > > "fields":{ > > > > "body":{ > > > > > > > > ``` > > > > } > > > > } > > > > > > > > ``` > > > > > > > > }, > > > > "query":{ > > > > "dis\_max":{ > > > > "tie\_breaker":0.7, > > > > "queries":[ > > > > { > > > > "text":{ > > > > "body":{ > > > > "query":"Great Planes Rotor Blade", > > > > "type":"phrase" > > > > } > > > > } > > > > }, > > > > { > > > > "query\_string":{ > > > > "fields":[ > > > > "body" > > > > ], > > > > "query":"Great Planes Rotor Blade", > > > > "phrase\_slop":0, > > > > "minimum\_should\_match":"40%" > > > > } > > > > }, > > > > { > > > > "query\_string":{ > > > > "fields":[ > > > > "body" > > > > ], > > > > "query":"Great Planes Rotor Blade", > > > > "phrase\_slop":0, > > > > "minimum\_should\_match":"40%", > > > > "analyzer":"analyzer\_partial\_\*\*shingle" > > > > } > > > > } > > > > ] > > > > } > > > > } > > > > } > > > > > > > > Unfortunately, I'm getting results all over the place. Some items > > > > which use the word "blade" 4-5 times will rank higher than items that use > > > > the phrase "Great Planes" once. I assumed that shingling the query (using > > > > analyzer\_partial\_shingle) and then searching the indexed shingles would > > > > find "Great Planes" and increase the score, but it doesn't seem to be > > > > working that way. > > > > > > > > Anyone shed some light on what I'm doing wrong? > > > > > > > > -- -- ---
**Author:** ![polyfractal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/polyfractal/32/48162_2.png) [@polyfractal](https://discuss.elastic.co/u/polyfractal)\ **Post date:** [November 13, 2012, 6:10pm UTC](https://discuss.elastic.co/t/confused-about-shingle-behavior/9689/6 "2012-11-13T18:10:47Z")
Thanks for the help Igor, switching the query over to a "text" did the trick. Good to know about query\_string, I didn't realize it tokenized before analyzing. I'm not sure I understand why text\_phrase didn't work, however. It doesn't really matter now, but I'm curious so I can avoid that mistake in the future. Why do individual tokens match the shingle, but not a full phrase? -Zach On Monday, November 12, 2012 11:56:52 PM UTC-5, Zachary Tong wrote: > I'm really struggling to get proper shingle searching to work. I've tried > dozens of variations, using text, string\_query, bools, dis\_max. The whole > works. I simply cannot get it to function the way that I want. I imagine > I'm doing something fundamentally wrong, since this seems like an easy > behavior. My mapping looks like this: [ES Mapping · GitHub](https://gist.github.com/4063964) > > Basically, I'm indexing a field with a normal tokenizer as well as a > shingle tokenizer. With regards to search, I want to match exact phrases > first, then match shingled phrases next (e.g. partial phrases). I'm > searching for "Great Planes Rotor Blade" using the following query: > > { > "explain":true, > "size":5, > "from":0, > "highlight":{ > "pre\_tags":[ > "**" > ], > "post\_tags":[ > "**" > ], > "fields":{ > "body":{ > > ``` > } > } > > ``` > > }, > "query":{ > "dis\_max":{ > "tie\_breaker":0.7, > "queries":[ > { > "text":{ > "body":{ > "query":"Great Planes Rotor Blade", > "type":"phrase" > } > } > }, > { > "query\_string":{ > "fields":[ > "body" > ], > "query":"Great Planes Rotor Blade", > "phrase\_slop":0, > "minimum\_should\_match":"40%" > } > }, > { > "query\_string":{ > "fields":[ > "body" > ], > "query":"Great Planes Rotor Blade", > "phrase\_slop":0, > "minimum\_should\_match":"40%", > "analyzer":"analyzer\_partial\_shingle" > } > } > ] > } > } > } > > Unfortunately, I'm getting results all over the place. Some items which > use the word "blade" 4-5 times will rank higher than items that use the > phrase "Great Planes" once. I assumed that shingling the query (using > analyzer\_partial\_shingle) and then searching the indexed shingles would > find "Great Planes" and increase the score, but it doesn't seem to be > working that way. > > Anyone shed some light on what I'm doing wrong? -- ---
**Author:** ![Igor\_Motov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_motov/32/45193_2.png) [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)\ **Post date:** [November 13, 2012, 10:00pm UTC](https://discuss.elastic.co/t/confused-about-shingle-behavior/9689/7 "2012-11-13T22:00:11Z")
Yeah, not sure what's wrong with match\_phrase. It works for me though: [shingles · GitHub](https://gist.github.com/73fcd9f3e19165802e14) On Tuesday, November 13, 2012 1:10:47 PM UTC-5, Zachary Tong wrote: > Thanks for the help Igor, switching the query over to a "text" did the > trick. Good to know about query\_string, I didn't realize it tokenized > before analyzing. > > I'm not sure I understand why text\_phrase didn't work, however. It > doesn't really matter now, but I'm curious so I can avoid that mistake in > the future. Why do individual tokens match the shingle, but not a full > phrase? > > -Zach > > On Monday, November 12, 2012 11:56:52 PM UTC-5, Zachary Tong wrote: > > > I'm really struggling to get proper shingle searching to work. I've > > tried dozens of variations, using text, string\_query, bools, dis\_max. The > > whole works. I simply cannot get it to function the way that I want. I > > imagine I'm doing something fundamentally wrong, since this seems like an > > easy behavior. My mapping looks like this: > > [ES Mapping · GitHub](https://gist.github.com/4063964) > > > > Basically, I'm indexing a field with a normal tokenizer as well as a > > shingle tokenizer. With regards to search, I want to match exact phrases > > first, then match shingled phrases next (e.g. partial phrases). I'm > > searching for "Great Planes Rotor Blade" using the following query: > > > > { > > "explain":true, > > "size":5, > > "from":0, > > "highlight":{ > > "pre\_tags":[ > > "**" > > ], > > "post\_tags":[ > > "**" > > ], > > "fields":{ > > "body":{ > > > > ``` > > } > > } > > > > ``` > > > > }, > > "query":{ > > "dis\_max":{ > > "tie\_breaker":0.7, > > "queries":[ > > { > > "text":{ > > "body":{ > > "query":"Great Planes Rotor Blade", > > "type":"phrase" > > } > > } > > }, > > { > > "query\_string":{ > > "fields":[ > > "body" > > ], > > "query":"Great Planes Rotor Blade", > > "phrase\_slop":0, > > "minimum\_should\_match":"40%" > > } > > }, > > { > > "query\_string":{ > > "fields":[ > > "body" > > ], > > "query":"Great Planes Rotor Blade", > > "phrase\_slop":0, > > "minimum\_should\_match":"40%", > > "analyzer":"analyzer\_partial\_shingle" > > } > > } > > ] > > } > > } > > } > > > > Unfortunately, I'm getting results all over the place. Some items which > > use the word "blade" 4-5 times will rank higher than items that use the > > phrase "Great Planes" once. I assumed that shingling the query (using > > analyzer\_partial\_shingle) and then searching the indexed shingles would > > find "Great Planes" and increase the score, but it doesn't seem to be > > working that way. > > > > Anyone shed some light on what I'm doing wrong? -- ---
**Author:** ![polyfractal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/polyfractal/32/48162_2.png) [@polyfractal](https://discuss.elastic.co/u/polyfractal)\ **Post date:** [November 13, 2012, 10:06pm UTC](https://discuss.elastic.co/t/confused-about-shingle-behavior/9689/8 "2012-11-13T22:06:46Z")
Weird. I am running a few versions behind (0.19.8), where \*text \*hasn't been replaced with _match_ yet, so perhaps it's due to that. Or something else that I have wrong somewhere else. Either way, thanks for the help, really appreciate it =) On Tuesday, November 13, 2012 5:00:11 PM UTC-5, Igor Motov wrote: > Yeah, not sure what's wrong with match\_phrase. It works for me though: > [shingles · GitHub](https://gist.github.com/73fcd9f3e19165802e14) > > On Tuesday, November 13, 2012 1:10:47 PM UTC-5, Zachary Tong wrote: > > > Thanks for the help Igor, switching the query over to a "text" did the > > trick. Good to know about query\_string, I didn't realize it tokenized > > before analyzing. > > > > I'm not sure I understand why text\_phrase didn't work, however. It > > doesn't really matter now, but I'm curious so I can avoid that mistake in > > the future. Why do individual tokens match the shingle, but not a full > > phrase? > > > > -Zach > > > > On Monday, November 12, 2012 11:56:52 PM UTC-5, Zachary Tong wrote: > > > > > I'm really struggling to get proper shingle searching to work. I've > > > tried dozens of variations, using text, string\_query, bools, dis\_max. The > > > whole works. I simply cannot get it to function the way that I want. I > > > imagine I'm doing something fundamentally wrong, since this seems like an > > > easy behavior. My mapping looks like this: > > > [ES Mapping · GitHub](https://gist.github.com/4063964) > > > > > > Basically, I'm indexing a field with a normal tokenizer as well as a > > > shingle tokenizer. With regards to search, I want to match exact phrases > > > first, then match shingled phrases next (e.g. partial phrases). I'm > > > searching for "Great Planes Rotor Blade" using the following query: > > > > > > { > > > "explain":true, > > > "size":5, > > > "from":0, > > > "highlight":{ > > > "pre\_tags":[ > > > "**" > > > ], > > > "post\_tags":[ > > > "**" > > > ], > > > "fields":{ > > > "body":{ > > > > > > ``` > > > } > > > } > > > > > > ``` > > > > > > }, > > > "query":{ > > > "dis\_max":{ > > > "tie\_breaker":0.7, > > > "queries":[ > > > { > > > "text":{ > > > "body":{ > > > "query":"Great Planes Rotor Blade", > > > "type":"phrase" > > > } > > > } > > > }, > > > { > > > "query\_string":{ > > > "fields":[ > > > "body" > > > ], > > > "query":"Great Planes Rotor Blade", > > > "phrase\_slop":0, > > > "minimum\_should\_match":"40%" > > > } > > > }, > > > { > > > "query\_string":{ > > > "fields":[ > > > "body" > > > ], > > > "query":"Great Planes Rotor Blade", > > > "phrase\_slop":0, > > > "minimum\_should\_match":"40%", > > > "analyzer":"analyzer\_partial\_shingle" > > > } > > > } > > > ] > > > } > > > } > > > } > > > > > > Unfortunately, I'm getting results all over the place. Some items which > > > use the word "blade" 4-5 times will rank higher than items that use the > > > phrase "Great Planes" once. I assumed that shingling the query (using > > > analyzer\_partial\_shingle) and then searching the indexed shingles would > > > find "Great Planes" and increase the score, but it doesn't seem to be > > > working that way. > > > > > > Anyone shed some light on what I'm doing wrong? -- ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:04am UTC](https://discuss.elastic.co/t/confused-about-shingle-behavior/9689/9 "2017-07-06T03:04:29Z")