# AllTermQueryParser? **URL:** \ **Category:** Elasticsearch\ **Created:** [June 12, 2013, 12:15pm UTC](https://discuss.elastic.co/t/alltermqueryparser/12378 "2013-06-12T12:15:17Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![clandestino\_bgd\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/clandestino_bgd_2/32/1315_2.png) [@clandestino\_bgd\_2](https://discuss.elastic.co/u/clandestino_bgd_2)\ **Post date:** [June 12, 2013, 12:15pm UTC](https://discuss.elastic.co/t/alltermqueryparser/12378/1 "2013-06-12T12:15:17Z")
Hi, I am looking for the way how to use token payloads in ES, pretty much same way as: in DelimitedPayloadTokenFilter, described at [http://searchhub.org/2009/08/05/getting-started-with-payloads/](http://searchhub.org/2009/08/05/getting-started-with-payloads/) I see in the ES codebase almost everything I think I need. package is: package org.elasticsearch.common.lucene.all There is AllTokenStream for indexing and AllTermQuery for querying, but I guess AllTermsQueryParser is missing somehow, or I am missing something? In other words in my query I need to instruct ES to use AllTermQuery and for that I understand I need AllTermQueryParser, but there is no such thing. My questions are: 1. Is there any reason why such query parser is not implemented? 2. What would be the best way to implement such query parser? 3. Is there any better way to implement this use case? Thanks, Milan -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![clandestino\_bgd\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/clandestino_bgd_2/32/1315_2.png) [@clandestino\_bgd\_2](https://discuss.elastic.co/u/clandestino_bgd_2)\ **Post date:** [June 13, 2013, 11:39am UTC](https://discuss.elastic.co/t/alltermqueryparser/12378/2 "2013-06-13T11:39:44Z")
Hi again, still haven't found the answer on how to tell ES to use: AllTokenStream in the analysis phase and AllTermQuery in the query dsl so that: DOC1 has :content" field: "The quick|2.0 red|2.0 fox|10.0 jumped|5.0 over the lazy|2.0 brown|2.0 dogs|10.0" DOC2 has "content" field: "The quick|3.0 red|2.0 fox|10.0 jumped|5.0 over the lazy|2.0 brown|2.0 dogs|10.0" float values (score boosts) are added in the token payload. I understand that AllTokenStream does that, but how do I tell that in index call? Then what do I need to do to get DOC2 scored better than DOC1 when I search for "red" in "content field"? I understand I need to use AllTermQuery, but how do I do that? Thanks in advance, Milan On Wed, Jun 12, 2013 at 2:15 PM, Milan Agatonovic \< [milan.agatonovic@gmail.com](mailto:milan.agatonovic@gmail.com)\> wrote: > Hi, > I am looking for the way how to use token payloads in ES, pretty much same > way as: > in DelimitedPayloadTokenFilter, described > at [http://searchhub.org/2009/08/05/getting-started-with-payloads/](http://searchhub.org/2009/08/05/getting-started-with-payloads/) > > I see in the ES codebase almost everything I think I need. > package is: package org.elasticsearch.common.lucene.all > > There is AllTokenStream for indexing and AllTermQuery for querying, but I > guess AllTermsQueryParser is missing somehow, or I am missing something? > > In other words in my query I need to instruct ES to use AllTermQuery and > for that I understand I need AllTermQueryParser, but there is no such thing. > > My questions are: > > 1. Is there any reason why such query parser is not implemented? > 2. What would be the best way to implement such query parser? > 3. Is there any better way to implement this use case? > > Thanks, > Milan -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [June 13, 2013, 9:22pm UTC](https://discuss.elastic.co/t/alltermqueryparser/12378/3 "2013-06-13T21:22:48Z")
I do not see the relationship between the DelimitedPayloadTokeFilter and the AllTermQuery except for payloads. The lucene.all package contains classes used for the all field. [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/mapping/all-field/) The all field uses the standard query parser. The AllFieldQuery is the only elasticsearch query that uses payloads. I often thought about how to incorporate payloads with elasticsearch, which I think is difficult because what would the JSON representation look like? You can create a plugin to create a new query/queryparser in Elasticsearch, although you cannot add it to the existing QueryBuilders static imports. Cheers, Ivan On Wed, Jun 12, 2013 at 5:15 AM, Milan Agatonovic \< [milan.agatonovic@gmail.com](mailto:milan.agatonovic@gmail.com)\> wrote: > Hi, > I am looking for the way how to use token payloads in ES, pretty much same > way as: > in DelimitedPayloadTokenFilter, described > at [http://searchhub.org/2009/08/05/getting-started-with-payloads/](http://searchhub.org/2009/08/05/getting-started-with-payloads/) > > I see in the ES codebase almost everything I think I need. > package is: package org.elasticsearch.common.lucene.all > > There is AllTokenStream for indexing and AllTermQuery for querying, but I > guess AllTermsQueryParser is missing somehow, or I am missing something? > > In other words in my query I need to instruct ES to use AllTermQuery and > for that I understand I need AllTermQueryParser, but there is no such thing. > > My questions are: > > 1. Is there any reason why such query parser is not implemented? > 2. What would be the best way to implement such query parser? > 3. Is there any better way to implement this use case? > > Thanks, > Milan > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![clandestino\_bgd\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/clandestino_bgd_2/32/1315_2.png) [@clandestino\_bgd\_2](https://discuss.elastic.co/u/clandestino_bgd_2)\ **Post date:** [June 14, 2013, 8:33am UTC](https://discuss.elastic.co/t/alltermqueryparser/12378/4 "2013-06-14T08:33:12Z")
Hi Ivan, thanks for the reply. Yes, I am certainly misled by the fact that AllTermQuery is the only query in ES that is using payloads. And by reading very sparse info on this forum (e.g here[http://elasticsearch-users.115913.n3.nabble.com/custom-similarity-setting-does-not-work-with-version-0-20-2-td4029500.html](http://elasticsearch-users.115913.n3.nabble.com/custom-similarity-setting-does-not-work-with-version-0-20-2-td4029500.html)) I was trying to find the way how to add Lucene's PayloadTermQuery into ES, but I couldn't find the way apart from adding it to the source code. I could not find any plugin that creates Custom Query Parser and registers it to ES. Is there any? Then I read this[https://groups.google.com/forum/?fromgroups#!topic/elasticsearch/GXUxysJ8BO4](https://groups.google.com/forum/?fromgroups#!topic/elasticsearch/GXUxysJ8BO4) where it was said that you can basically use AllTermQuery to make your boosts in payload work. Also when I compare PayloadTermQuery and AllTermQuery they look pretty similar, so I thought if I can "tell" ES to use AllTermQuery when I need overall score to be calculated with the values in my payload, I'll do fine. From the indexing perspective I think I have everything. Custom token filter and analyzer work through analyze API. I also created custom similarity (as per posts above) but currently I do not know how to use it. For example both Lucene's PayloadTermQuery and AllTermQuery have their own embedded extension of SpanScorer to scorePayload. Any help would be much appreciated. Thanks, MIlan On Thu, Jun 13, 2013 at 11:22 PM, Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com) wrote: > I do not see the relationship between the DelimitedPayloadTokeFilter and > the AllTermQuery except for payloads. The lucene.all package contains > classes used for the all field. > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/mapping/all-field/) The all > field uses the standard query parser. > > The AllFieldQuery is the only elasticsearch query that uses payloads. I > often thought about how to incorporate payloads with elasticsearch, which I > think is difficult because what would the JSON representation look like? > > You can create a plugin to create a new query/queryparser in > Elasticsearch, although you cannot add it to the existing QueryBuilders > static imports. > > Cheers, > > Ivan > > On Wed, Jun 12, 2013 at 5:15 AM, Milan Agatonovic \< > [milan.agatonovic@gmail.com](mailto:milan.agatonovic@gmail.com)\> wrote: > > > Hi, > > I am looking for the way how to use token payloads in ES, pretty much > > same way as: > > in DelimitedPayloadTokenFilter, described > > at [http://searchhub.org/2009/08/05/getting-started-with-payloads/](http://searchhub.org/2009/08/05/getting-started-with-payloads/) > > > > I see in the ES codebase almost everything I think I need. > > package is: package org.elasticsearch.common.lucene.all > > > > There is AllTokenStream for indexing and AllTermQuery for querying, but I > > guess AllTermsQueryParser is missing somehow, or I am missing something? > > > > In other words in my query I need to instruct ES to use AllTermQuery and > > for that I understand I need AllTermQueryParser, but there is no such thing. > > > > My questions are: > > > > 1. Is there any reason why such query parser is not implemented? > > 2. What would be the best way to implement such query parser? > > 3. Is there any better way to implement this use case? > > > > Thanks, > > Milan > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Devaraja\_Swami](https://avatars.discourse-cdn.com/v4/letter/d/b5a626/32.png) [@Devaraja\_Swami](https://discuss.elastic.co/u/Devaraja_Swami)\ **Post date:** [March 17, 2015, 6:04am UTC](https://discuss.elastic.co/t/alltermqueryparser/12378/5 "2015-03-17T06:04:37Z")
Clandestino, did you have any resolution for this? If so, would be deeply grateful for any insights/solutions you can provide. Regards, devarajaswami On Friday, June 14, 2013 at 1:33:12 AM UTC-7, clandestino\_bgd wrote: > Hi Ivan, > thanks for the reply. > Yes, I am certainly misled by the fact that AllTermQuery is the only > query in ES that is using payloads. > And by reading very sparse info on this forum (e.g here > [http://elasticsearch-users.115913.n3.nabble.com/custom-similarity-setting-does-not-work-with-version-0-20-2-td4029500.html](http://elasticsearch-users.115913.n3.nabble.com/custom-similarity-setting-does-not-work-with-version-0-20-2-td4029500.html)) > I was trying to find the way how to add Lucene's PayloadTermQuery into ES, > but I couldn't find the way apart from adding it to the source code. I > could not find any plugin that creates Custom Query Parser and registers it > to ES. > Is there any? > > Then I read this > [https://groups.google.com/forum/?fromgroups#!topic/elasticsearch/GXUxysJ8BO4](https://groups.google.com/forum/?fromgroups#!topic/elasticsearch/GXUxysJ8BO4) where > it was said that you can basically use AllTermQuery to make your boosts in > payload work. > Also when I compare PayloadTermQuery and AllTermQuery they look pretty similar, > so I thought if I can "tell" ES to use AllTermQuery when I need overall > score to be calculated with the values in my payload, I'll do fine. > > From the indexing perspective I think I have everything. Custom token > filter and analyzer work through analyze API. I also created custom > similarity (as per posts above) but currently I do not know how to use it. > For example both Lucene's PayloadTermQuery and AllTermQuery have their > own embedded extension of SpanScorer to scorePayload. > > Any help would be much appreciated. > Thanks, > MIlan > > On Thu, Jun 13, 2013 at 11:22 PM, Ivan Brusic \<[iv...@brusic.com](mailto:iv...@brusic.com) > \\> wrote: > > > I do not see the relationship between the DelimitedPayloadTokeFilter and > > the AllTermQuery except for payloads. The lucene.all package contains > > classes used for the all field. > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/mapping/all-field/) The all > > field uses the standard query parser. > > > > The AllFieldQuery is the only elasticsearch query that uses payloads. I > > often thought about how to incorporate payloads with elasticsearch, which I > > think is difficult because what would the JSON representation look like? > > > > You can create a plugin to create a new query/queryparser in > > Elasticsearch, although you cannot add it to the existing QueryBuilders > > static imports. > > > > Cheers, > > > > Ivan > > > > On Wed, Jun 12, 2013 at 5:15 AM, Milan Agatonovic \<[milan.ag...@gmail.com](mailto:milan.ag...@gmail.com) > > \\> wrote: > > > > > Hi, > > > I am looking for the way how to use token payloads in ES, pretty much > > > same way as: > > > in DelimitedPayloadTokenFilter, described > > > at [http://searchhub.org/2009/08/05/getting-started-with-payloads/](http://searchhub.org/2009/08/05/getting-started-with-payloads/) > > > > > > I see in the ES codebase almost everything I think I need. > > > package is: package org.elasticsearch.common.lucene.all > > > > > > There is AllTokenStream for indexing and AllTermQuery for querying, but > > > I guess AllTermsQueryParser is missing somehow, or I am missing something? > > > > > > In other words in my query I need to instruct ES to use AllTermQuery and > > > for that I understand I need AllTermQueryParser, but there is no such thing. > > > > > > My questions are: > > > > > > 1. Is there any reason why such query parser is not implemented? > > > 2. What would be the best way to implement such query parser? > > > 3. Is there any better way to implement this use case? > > > > > > Thanks, > > > Milan > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/cd86ed25-3655-431c-9b66-7cc70ef262bb%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/cd86ed25-3655-431c-9b66-7cc70ef262bb%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 12:26am UTC](https://discuss.elastic.co/t/alltermqueryparser/12378/6 "2017-07-06T00:26:22Z")