# Match text ending in a word **URL:** \ **Category:** Elasticsearch\ **Created:** [October 11, 2012, 8:59pm UTC](https://discuss.elastic.co/t/match-text-ending-in-a-word/9329 "2012-10-11T20:59:41Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Mike](https://avatars.discourse-cdn.com/v4/letter/m/58f4c7/32.png) [@Mike](https://discuss.elastic.co/u/Mike)\ **Post date:** [October 11, 2012, 8:59pm UTC](https://discuss.elastic.co/t/match-text-ending-in-a-word/9329/1 "2012-10-11T20:59:41Z")
Is there a way to search for text that ends with a word? For example, if I wanted to match text that ended in "control", "pollution control" would match but "control price" would not. -- ---
**Author:** ![Mike](https://avatars.discourse-cdn.com/v4/letter/m/58f4c7/32.png) [@Mike](https://discuss.elastic.co/u/Mike)\ **Post date:** [October 11, 2012, 9:03pm UTC](https://discuss.elastic.co/t/match-text-ending-in-a-word/9329/2 "2012-10-11T21:03:25Z")
I imagine the only way to do this would be to use a regex query, which isn't exposed in Elastic at the moment due to it's poor performance? On Thursday, October 11, 2012 4:59:41 PM UTC-4, Mike wrote: > Is there a way to search for text that ends with a word? > For example, if I wanted to match text that ended in "control", "pollution > control" would match but "control price" would not. -- ---
**Author:** ![Chris\_Male](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chris_male/32/2607_2.png) [@Chris\_Male](https://discuss.elastic.co/u/Chris_Male)\ **Post date:** [October 11, 2012, 10:50pm UTC](https://discuss.elastic.co/t/match-text-ending-in-a-word/9329/3 "2012-10-11T22:50:48Z")
Mike, Thinking of a couple of solutions. One way would be to use some custom analysis to mark the last word in your text some way, either by adding a payload or just adding a prefix/suffix like \_last. You could then rewrite your queries to make use of this mark. Alternatively (and far more complex) you could create your own SpanPositionCheckQuery implementation which checked the position of the matches. Good luck. On Friday, October 12, 2012 10:03:25 AM UTC+13, Mike wrote: > I imagine the only way to do this would be to use a regex query, which > isn't exposed in Elastic at the moment due to it's poor performance? > > On Thursday, October 11, 2012 4:59:41 PM UTC-4, Mike wrote: > > > Is there a way to search for text that ends with a word? > > For example, if I wanted to match text that ended in "control", > > "pollution control" would match but "control price" would not. -- ---
**Author:** ![simonw\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simonw_2/32/1130_2.png) [@simonw\_2](https://discuss.elastic.co/u/simonw_2)\ **Post date:** [October 12, 2012, 8:29am UTC](https://discuss.elastic.co/t/match-text-ending-in-a-word/9329/4 "2012-10-12T08:29:32Z")
another solution is to index your text in a second field and drop everything but the last word. this obviously only works if you are not trying to do this on free-text but on titles or something like this. ie. "pollution control price" would be tricky 😉 simon On Friday, October 12, 2012 12:50:49 AM UTC+2, Chris Male wrote: > Mike, > > Thinking of a couple of solutions. One way would be to use some custom > analysis to mark the last word in your text some way, either by adding a > payload or just adding a prefix/suffix like \_last. You could then rewrite > your queries to make use of this mark. > > Alternatively (and far more complex) you could create your own > SpanPositionCheckQuery implementation which checked the position of the > matches. > > Good luck. > > On Friday, October 12, 2012 10:03:25 AM UTC+13, Mike wrote: > > > I imagine the only way to do this would be to use a regex query, which > > isn't exposed in Elastic at the moment due to it's poor performance? > > > > On Thursday, October 11, 2012 4:59:41 PM UTC-4, Mike wrote: > > > > > Is there a way to search for text that ends with a word? > > > For example, if I wanted to match text that ended in "control", > > > "pollution control" would match but "control price" would not. -- ---
**Author:** ![phill](https://avatars.discourse-cdn.com/v4/letter/p/779978/32.png) [@phill](https://discuss.elastic.co/u/phill)\ **Post date:** [October 15, 2012, 6:42pm UTC](https://discuss.elastic.co/t/match-text-ending-in-a-word/9329/5 "2012-10-15T18:42:05Z")
Simon, Why do you think indexing the last word would be tricky? It seems to me that the last word in either the body or the title of a doc seems easy to identify. A more general solution which Mike didn't say he needed, is to index the end of the text in _reverse_. Using Simon's example "ecirp lortnoc noitullop". You only have to index as much of the end as you allow the user to search. Then you take whatever the user wants to find, reverse it, and find it in this special field. You can do this using a prefix query which should be a lot faster then trying a RE on every value of a field and a lot easier to write then a special span query or adding a payload. Chris, How would adding a payload just to mark one word in the text be useful? Forget about passing it down through ES to Lucene for a moment, How could I even build a Lucene query tree that says "and term X is exactly the last word." which is really not just as easy to generate in the code as adding an extra sub-expression in the query that simply says "last\_word":"price" (to re-use Simon's example). This new "ends\_with" field is a case where it should not be part of the "all" field. If it is just the last word and not the last part of the document, you also don't need analysis and positions etc. thus the overhead for the new field seems both reasonable and even similar to adding just enough payload information to identify the special position of a word. An example I can think of that might require special payloads is answering the question: Did the hit highlight come from the very end of the body of text? That _would_ require some rewriting, but even then I think all the information is already available in the index it is more of a problem of find it. To me this new field is also an example of indexing to support answering (as fast as possible) a question that you know you need without worrying about de-normalizing. A lot of the design of an index is about pre-calculating the values to support the searches and reducing search time overhead. -Paul On 10/12/2012 1:29 AM, simonw wrote: > another solution is to index your text in a second field and drop > everything but the last word. this obviously only works if you are not > trying to do this on free-text but on titles or something like this. > ie. "pollution control price" would be tricky 😉 > > simon -- ---
**Author:** ![simonw\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simonw_2/32/1130_2.png) [@simonw\_2](https://discuss.elastic.co/u/simonw_2)\ **Post date:** [October 15, 2012, 6:57pm UTC](https://discuss.elastic.co/t/match-text-ending-in-a-word/9329/6 "2012-10-15T18:57:44Z")
Hey Paul, I might have not been explicit enough in my last post. What I meant by tricky was if you (reusing my own example 🙂 "pollution control price" the term "control" is a successor of "pollution" and a prefix of "price" so if you want to solve it here you can't just index the last term ie. this get's a lot trickier. If that is not the case the second field with only the last terms is as you said the "fast" and "simple" solution. I hope that clarifies this a bit. simon On Monday, October 15, 2012 8:41:59 PM UTC+2, P Hill wrote: > Simon, > > Why do you think indexing the last word would be tricky? > It seems to me that the last word in either the body or the title of a > doc seems easy to identify. > > A more general solution which Mike didn't say he needed, is to index the > end of the text in _reverse_. Using Simon's example "ecirp lortnoc > noitullop". You only have to index as much of the end as you allow the > user to search. Then you take whatever the user wants to find, reverse > it, and find it in this special field. You can do this using a prefix > query which should be a lot faster then trying a RE on every value of a > field and a lot easier to write then a special span query or adding a > payload. > > Chris, How would adding a payload just to mark one word in the text be > useful? Forget about passing it down through ES to Lucene for a > moment, How could I even build a Lucene query tree that says "and term > X is exactly the last word." which is really not just as easy to > generate in the code as adding an extra sub-expression in the query that > simply says "last\_word":"price" (to re-use Simon's example). > > This new "ends\_with" field is a case where it should not be part of the > "all" field. > > If it is just the last word and not the last part of the document, you > also don't need analysis and positions etc. thus the overhead for the > new field seems both reasonable and even similar to adding just enough > payload information to identify the special position of a word. An > example I can think of that might require special payloads is answering > the question: Did the hit highlight come from the very end of the body > of text? That _would_ require some rewriting, but even then I think all > the information is already available in the index it is more of a > problem of find it. > > To me this new field is also an example of indexing to support answering > (as fast as possible) a question that you know you need without worrying > about de-normalizing. A lot of the design of an index is about > pre-calculating the values to support the searches and reducing search > time overhead. > > -Paul > > On 10/12/2012 1:29 AM, simonw wrote: > > > another solution is to index your text in a second field and drop > > everything but the last word. this obviously only works if you are not > > trying to do this on free-text but on titles or something like this. > > ie. "pollution control price" would be tricky 😉 > > > > simon -- ---
**Author:** ![Mike](https://avatars.discourse-cdn.com/v4/letter/m/58f4c7/32.png) [@Mike](https://discuss.elastic.co/u/Mike)\ **Post date:** [October 22, 2012, 9:22pm UTC](https://discuss.elastic.co/t/match-text-ending-in-a-word/9329/7 "2012-10-22T21:22:44Z")
Thanks a lot guys. Paul's solution works great for me. Just a note for anyone reading this, make sure you index the field as not\_analyzed or with the keyword analyzer, otherwise your prefix search will match the individual tokens in the text, defeating the entire purpose. On Monday, October 15, 2012 2:57:44 PM UTC-4, simonw wrote: > Hey Paul, > > I might have not been explicit enough in my last post. What I meant by > tricky was if you (reusing my own example 🙂 "pollution control price" the > term "control" is a successor of "pollution" and a prefix of "price" so if > you want to solve it here you can't just index the last term ie. this get's > a lot trickier. If that is not the case the second field with only the last > terms is as you said the "fast" and "simple" solution. > > I hope that clarifies this a bit. > > simon > > On Monday, October 15, 2012 8:41:59 PM UTC+2, P Hill wrote: > > > Simon, > > > > Why do you think indexing the last word would be tricky? > > It seems to me that the last word in either the body or the title of a > > doc seems easy to identify. > > > > A more general solution which Mike didn't say he needed, is to index the > > end of the text in _reverse_. Using Simon's example "ecirp lortnoc > > noitullop". You only have to index as much of the end as you allow the > > user to search. Then you take whatever the user wants to find, reverse > > it, and find it in this special field. You can do this using a prefix > > query which should be a lot faster then trying a RE on every value of a > > field and a lot easier to write then a special span query or adding a > > payload. > > > > Chris, How would adding a payload just to mark one word in the text be > > useful? Forget about passing it down through ES to Lucene for a > > moment, How could I even build a Lucene query tree that says "and term > > X is exactly the last word." which is really not just as easy to > > generate in the code as adding an extra sub-expression in the query that > > simply says "last\_word":"price" (to re-use Simon's example). > > > > This new "ends\_with" field is a case where it should not be part of the > > "all" field. > > > > If it is just the last word and not the last part of the document, you > > also don't need analysis and positions etc. thus the overhead for the > > new field seems both reasonable and even similar to adding just enough > > payload information to identify the special position of a word. An > > example I can think of that might require special payloads is answering > > the question: Did the hit highlight come from the very end of the body > > of text? That _would_ require some rewriting, but even then I think all > > the information is already available in the index it is more of a > > problem of find it. > > > > To me this new field is also an example of indexing to support answering > > (as fast as possible) a question that you know you need without worrying > > about de-normalizing. A lot of the design of an index is about > > pre-calculating the values to support the searches and reducing search > > time overhead. > > > > -Paul > > > > On 10/12/2012 1:29 AM, simonw wrote: > > > > > another solution is to index your text in a second field and drop > > > everything but the last word. this obviously only works if you are not > > > trying to do this on free-text but on titles or something like this. > > > ie. "pollution control price" would be tricky 😉 > > > > > > simon -- ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:07am UTC](https://discuss.elastic.co/t/match-text-ending-in-a-word/9329/8 "2017-07-06T03:07:32Z")