# Reverse search: I give you a block of text, you tell me which indexed documents have a specific field value that matches it **URL:** \ **Category:** Elasticsearch\ **Created:** [February 20, 2013, 10:58pm UTC](https://discuss.elastic.co/t/reverse-search-i-give-you-a-block-of-text-you-tell-me-which-indexed-documents-have-a-specific-field-value-that-matches-it/10830 "2013-02-20T22:58:35Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![Jeremy\_Jongsma\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jeremy_jongsma_2/32/2098_2.png) [@Jeremy\_Jongsma\_2](https://discuss.elastic.co/u/Jeremy_Jongsma_2)\ **Post date:** [February 20, 2013, 10:58pm UTC](https://discuss.elastic.co/t/reverse-search-i-give-you-a-block-of-text-you-tell-me-which-indexed-documents-have-a-specific-field-value-that-matches-it/10830/1 "2013-02-20T22:58:35Z")
I have an elasticsearch index of stock and future symbols (around 100,000 right now). Each document contains at minimum the ticker symbol ("GOOG"), a description/company name ("Google"), and optionally some other metadata like keywords and industry categories. I'd like to be able to analyze the text of a news story and determine which symbols, if any, it is likely to be related to. This can be because the symbol is specifically mentioned in the story, the company name is mentioned in the story, or specific keywords or industries are mentioned. So, INPUT: story text OUTPUT: list of symbols and scores representing the likelihood they are related to this story (GOOG: 96%, MSFT: 43%, etc) Is there anything in elasticsearch that will help me here? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [February 20, 2013, 11:19pm UTC](https://discuss.elastic.co/t/reverse-search-i-give-you-a-block-of-text-you-tell-me-which-indexed-documents-have-a-specific-field-value-that-matches-it/10830/2 "2013-02-20T23:19:47Z")
This use case is definitely up Jorg's alley. You are better off analyzing the text beforehand using some entity extraction tool. There is nothing in Elasticsearch that would help you off the shelf, but there are a few in Lucene, so maybe a plugin can be created. After that you can use a dismax query with the different terms, but you won't get relevancy percentages, just relative scores. Hopefully Jorg will chime in. -- Ivan On Wed, Feb 20, 2013 at 2:58 PM, Jeremy Jongsma [jeremy@jongsma.org](mailto:jeremy@jongsma.org) wrote: > I have an elasticsearch index of stock and future symbols (around 100,000 > right now). Each document contains at minimum the ticker symbol ("GOOG"), a > description/company name ("Google"), and optionally some other metadata > like keywords and industry categories. > > I'd like to be able to analyze the text of a news story and determine > which symbols, if any, it is likely to be related to. This can be because > the symbol is specifically mentioned in the story, the company name is > mentioned in the story, or specific keywords or industries are mentioned. > > So, > > INPUT: story text > OUTPUT: list of symbols and scores representing the likelihood they are > related to this story (GOOG: 96%, MSFT: 43%, etc) > > Is there anything in elasticsearch that will help me here? > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [February 20, 2013, 11:46pm UTC](https://discuss.elastic.co/t/reverse-search-i-give-you-a-block-of-text-you-tell-me-which-indexed-documents-have-a-specific-field-value-that-matches-it/10830/3 "2013-02-20T23:46:19Z")
Jeremy, you are right to ask if ES can help in such cases. In fact, it can help theoretically, but at the moment, you have to prepare a lot of work for yourself before you can take advantage of the strength of ES (searching in a vast amount of data). If you know the key words in advance, you could just throw your data over the fence and let ES pick the countings of how many occurences are there, and you can do filtering or faceting to find out how many documents are involved. By presenting such results in lists or tables, you could extract some relationship analysis. But in general, to compute numbers about yet unknown relationships among words and terms in large numbers of unstructured texts, you enter the area of text mining, or natural language processing (NLP). Tools like UIMA, OpenNLP, and Stanford NLP exist that provide all the math and hard work behind such analysis. ES does not do that kind of analysis out of the box, but with plugins, it could be extended to integrate NLP tools. On the Lucene level, ES can carry text annotations with the payload mechanism into the index. Such plugins do not exist yet (well I don't know of any), I tried a bit for myself with UIMA, OpenNLP, and Stanford NLP, but I got stuck how to deal with the Lucene payload mechanism, since it requires designing the query and presentation of payloads very carefully. Just a heads up, I think, when reading the comments around the 24m$ funding, that out-of-the-box text mining and easy-to-use NLP will get more into focus of ES in the future. Best regards, Jörg On Thursday, February 21, 2013 12:19:47 AM UTC+1, Ivan Brusic wrote: > This use case is definitely up Jorg's alley. > > You are better off analyzing the text beforehand using some entity > extraction tool. There is nothing in Elasticsearch that would help you off > the shelf, but there are a few in Lucene, so maybe a plugin can be created. > After that you can use a dismax query with the different terms, but you > won't get relevancy percentages, just relative scores. Hopefully Jorg will > chime in. > > -- > Ivan > > On Wed, Feb 20, 2013 at 2:58 PM, Jeremy Jongsma \<[jer...@jongsma.org](mailto:jer...@jongsma.org)\ > > > wrote: > > > I have an elasticsearch index of stock and future symbols (around 100,000 > > right now). Each document contains at minimum the ticker symbol ("GOOG"), a > > description/company name ("Google"), and optionally some other metadata > > like keywords and industry categories. > > > > I'd like to be able to analyze the text of a news story and determine > > which symbols, if any, it is likely to be related to. This can be because > > the symbol is specifically mentioned in the story, the company name is > > mentioned in the story, or specific keywords or industries are mentioned. > > > > So, > > > > INPUT: story text > > OUTPUT: list of symbols and scores representing the likelihood they are > > related to this story (GOOG: 96%, MSFT: 43%, etc) > > > > Is there anything in elasticsearch that will help me here? > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![karmi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/karmi/32/44951_2.png) [@karmi](https://discuss.elastic.co/u/karmi)\ **Post date:** [February 21, 2013, 8:35am UTC](https://discuss.elastic.co/t/reverse-search-i-give-you-a-block-of-text-you-tell-me-which-indexed-documents-have-a-specific-field-value-that-matches-it/10830/4 "2013-02-21T08:35:54Z")
> INPUT: story text > OUTPUT: list of symbols and scores representing the likelihood they are > related to this story (GOOG: 96%, MSFT: 43%, etc) > > Is there anything in elasticsearch that will help me here? Absolutely -- look at the Percolator API: [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/percolate.html). If I understand you correctly, you would register your documents as queries with percolator, and then ask the percolator on which of these queries a specific document matches. You will _not_ get scoring info ("GOOG: 96%") in the percolator output itself, but you can get creative with the registered queries, of course (using `min_score` and custom boosting etc). Karel -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [February 21, 2013, 8:43am UTC](https://discuss.elastic.co/t/reverse-search-i-give-you-a-block-of-text-you-tell-me-which-indexed-documents-have-a-specific-field-value-that-matches-it/10830/5 "2013-02-21T08:43:41Z")
What I would probably do here is to simply use a Match Query with your full text against your fields. If my answer is incorrect, I probably don't understand what you are looking for. Can you illustrate your use case with one of your indexed documents and one text you want to search for? -- David Pilato | Technical Advocate | [Elasticsearch.com](http://Elasticsearch.com) @dadoonet | @elasticsearchfr | @scrutmydocs Le 21 févr. 2013 à 09:35, Karel Minařík [karel.minarik@gmail.com](mailto:karel.minarik@gmail.com) a écrit : > INPUT: story text > OUTPUT: list of symbols and scores representing the likelihood they are related to this story (GOOG: 96%, MSFT: 43%, etc) > > Is there anything in elasticsearch that will help me here? > > Absolutely -- look at the Percolator API: [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/percolate.html). > > If I understand you correctly, you would register your documents as queries with percolator, and then ask the percolator on which of these queries a specific document matches. You will _not_ get scoring info ("GOOG: 96%") in the percolator output itself, but you can get creative with the registered queries, of course (using `min_score` and custom boosting etc). > > Karel > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Jeremy\_Jongsma\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jeremy_jongsma_2/32/2098_2.png) [@Jeremy\_Jongsma\_2](https://discuss.elastic.co/u/Jeremy_Jongsma_2)\ **Post date:** [February 21, 2013, 4:58pm UTC](https://discuss.elastic.co/t/reverse-search-i-give-you-a-block-of-text-you-tell-me-which-indexed-documents-have-a-specific-field-value-that-matches-it/10830/6 "2013-02-21T16:58:13Z")
Thanks for the info Ivan and Jorg. It sounds like my best bet currently is to do my own term extraction and then run the terms through a Dis Max query against the symbol index like Ivan suggested. For those interested, here is a more detailed explanation of my use case: Say I have the following documents in my symbol index: { "symbol": "GOOG", "description": "Google Inc." } { "symbol": "YHOO", "description": "Yahoo! Inc." } { "symbol": "MSFT", "description": "Microsoft Corp." } I then receive the following news story text: "Mayer, who spent 13 years helping to build Google into the Internet's most powerful company, has vowed to revive Yahoo Inc.'s revenue growth by establishing more of the company's services as daily habits that "delight and inspire" their users." I want to build a library that can process that text and tell me there is a high probability that the symbols ["GOOG", "YHOO"] are applicable to it. On Thu, Feb 21, 2013 at 2:43 AM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: > What I would probably do here is to simply use a Match Query with your > full text against your fields. > > If my answer is incorrect, I probably don't understand what you are > looking for. > Can you illustrate your use case with one of your indexed documents and > one text you want to search for? > > -- > _David Pilato_ | _Technical Advocate_ | _[Elasticsearch.com](http://Elasticsearch.com)_ > @dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr[https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr) > | @scrutmydocs [https://twitter.com/scrutmydocs](https://twitter.com/scrutmydocs) > > Le 21 févr. 2013 à 09:35, Karel Minařík [karel.minarik@gmail.com](mailto:karel.minarik@gmail.com) a écrit > : > > INPUT: story text > > > OUTPUT: list of symbols and scores representing the likelihood they are > > related to this story (GOOG: 96%, MSFT: 43%, etc) > > > > Is there anything in elasticsearch that will help me here? > > Absolutely -- look at the Percolator API: > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/percolate.html). > > If I understand you correctly, you would register your documents as > queries with percolator, and then ask the percolator on which of these > queries a specific document matches. You will _not_ get scoring info > ("GOOG: 96%") in the percolator output itself, but you can get creative > with the registered queries, of course (using `min_score` and custom > boosting etc). > > Karel > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [February 21, 2013, 5:15pm UTC](https://discuss.elastic.co/t/reverse-search-i-give-you-a-block-of-text-you-tell-me-which-indexed-documents-have-a-specific-field-value-that-matches-it/10830/7 "2013-02-21T17:15:21Z")
The trickiness in your use case is the desired output. I would start off by using a simple queries with explain enabled and see if you can transform the resulting explain output into the format you seek, or perhaps simply use the scores. Scores are relative to the query and are not absolute. Start off simple. -- Ivan On Thu, Feb 21, 2013 at 8:58 AM, Jeremy Jongsma [jeremy@jongsma.org](mailto:jeremy@jongsma.org) wrote: > Thanks for the info Ivan and Jorg. It sounds like my best bet currently is > to do my own term extraction and then run the terms through a Dis Max query > against the symbol index like Ivan suggested. > > For those interested, here is a more detailed explanation of my use case: > > Say I have the following documents in my symbol index: > > { "symbol": "GOOG", "description": "Google Inc." } > { "symbol": "YHOO", "description": "Yahoo! Inc." } > { "symbol": "MSFT", "description": "Microsoft Corp." } > > I then receive the following news story text: > > "Mayer, who spent 13 years helping to build Google into the Internet's > most powerful company, has vowed to revive Yahoo Inc.'s revenue growth by > establishing more of the company's services as daily habits that "delight > and inspire" their users." > > I want to build a library that can process that text and tell me there is > a high probability that the symbols ["GOOG", "YHOO"] are applicable to it. > > On Thu, Feb 21, 2013 at 2:43 AM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: > > > What I would probably do here is to simply use a Match Query with your > > full text against your fields. > > > > If my answer is incorrect, I probably don't understand what you are > > looking for. > > Can you illustrate your use case with one of your indexed documents and > > one text you want to search for? > > > > -- > > _David Pilato_ | _Technical Advocate_ | _[Elasticsearch.com](http://Elasticsearch.com)_ > > @dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr[https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr) > > | @scrutmydocs [https://twitter.com/scrutmydocs](https://twitter.com/scrutmydocs) > > > > Le 21 févr. 2013 à 09:35, Karel Minařík [karel.minarik@gmail.com](mailto:karel.minarik@gmail.com) a > > écrit : > > > > INPUT: story text > > > > > OUTPUT: list of symbols and scores representing the likelihood they are > > > related to this story (GOOG: 96%, MSFT: 43%, etc) > > > > > > Is there anything in elasticsearch that will help me here? > > > > Absolutely -- look at the Percolator API: > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/percolate.html). > > > > If I understand you correctly, you would register your documents as > > queries with percolator, and then ask the percolator on which of these > > queries a specific document matches. You will _not_ get scoring info > > ("GOOG: 96%") in the percolator output itself, but you can get creative > > with the registered queries, of course (using `min_score` and custom > > boosting etc). > > > > Karel > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [February 21, 2013, 6:22pm UTC](https://discuss.elastic.co/t/reverse-search-i-give-you-a-block-of-text-you-tell-me-which-indexed-documents-have-a-specific-field-value-that-matches-it/10830/8 "2013-02-21T18:22:39Z")
Any interesting comments? I haven't seen any. One job position at Elasticsearch states: "We are re-branding our company including re-launching our website to optimize lead conversion. The new website will go live at the end of February," Hopefully the 0.21 release will also happen by the end of the month. Sorry for the thread jacking, Ivan On Wed, Feb 20, 2013 at 3:46 PM, Jörg Prante [joergprante@gmail.com](mailto:joergprante@gmail.com) wrote: > Just a heads up, I think, when reading the comments around the 24m$ > funding, that out-of-the-box text mining and easy-to-use NLP will get more > into focus of ES in the future. > > Best regards, > > Jörg -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [February 21, 2013, 6:48pm UTC](https://discuss.elastic.co/t/reverse-search-i-give-you-a-block-of-text-you-tell-me-which-indexed-documents-have-a-specific-field-value-that-matches-it/10830/9 "2013-02-21T18:48:25Z")
Mike Volpi at [http://www.indexventures.com/blog#post/603](http://www.indexventures.com/blog#post/603) "[...] Analytics are a tricky business because ultimately, the products end in two scenarios: (a) a skilled user to get the desired result from them; or (b) high degrees of structure that don’t answer the really big questions. We began to look elsewhere – what if you just wanted to “talk” to your data – sort of like how we have become accustomed to finding what we want as consumers from Google. [...]" and "[...] But, perhaps even more so is the vision of Elasticsearch. Through its peaks and valleys, Big Data is here to stay. Business and consumer experiences will never be the same because we will have vast amounts of information to make these experiences better. But, that vision will only come to be when we can talk to our data the way we talk to our own minds. Elasticsearch is a huge leap ahead in that direction." Google-like phrases in hope to find some analytic insight? Talk to data, like talk to own mind? Well, to be frankly, that's what natural language processing (NLP) technologies has been designed for. Jörg Am 21.02.13 19:22, schrieb Ivan Brusic: > Any interesting comments? I haven't seen any. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Itamar\_Syn\_Hershko](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/itamar_syn_hershko/32/725_2.png) [@Itamar\_Syn\_Hershko](https://discuss.elastic.co/u/Itamar_Syn_Hershko)\ **Post date:** [February 21, 2013, 7:02pm UTC](https://discuss.elastic.co/t/reverse-search-i-give-you-a-block-of-text-you-tell-me-which-indexed-documents-have-a-specific-field-value-that-matches-it/10830/10 "2013-02-21T19:02:05Z")
This is very similar to Lucene "MoreLikeThis" functionality. If you can get that block of text indexed, even temporarily, under a type in ES you could use MoreLikeThis to do this. On Thu, Feb 21, 2013 at 6:58 PM, Jeremy Jongsma [jeremy@jongsma.org](mailto:jeremy@jongsma.org) wrote: > Thanks for the info Ivan and Jorg. It sounds like my best bet currently is > to do my own term extraction and then run the terms through a Dis Max query > against the symbol index like Ivan suggested. > > For those interested, here is a more detailed explanation of my use case: > > Say I have the following documents in my symbol index: > > { "symbol": "GOOG", "description": "Google Inc." } > { "symbol": "YHOO", "description": "Yahoo! Inc." } > { "symbol": "MSFT", "description": "Microsoft Corp." } > > I then receive the following news story text: > > "Mayer, who spent 13 years helping to build Google into the Internet's > most powerful company, has vowed to revive Yahoo Inc.'s revenue growth by > establishing more of the company's services as daily habits that "delight > and inspire" their users." > > I want to build a library that can process that text and tell me there is > a high probability that the symbols ["GOOG", "YHOO"] are applicable to it. > > On Thu, Feb 21, 2013 at 2:43 AM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: > > > What I would probably do here is to simply use a Match Query with your > > full text against your fields. > > > > If my answer is incorrect, I probably don't understand what you are > > looking for. > > Can you illustrate your use case with one of your indexed documents and > > one text you want to search for? > > > > -- > > _David Pilato_ | _Technical Advocate_ | _[Elasticsearch.com](http://Elasticsearch.com)_ > > @dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr[https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr) > > | @scrutmydocs [https://twitter.com/scrutmydocs](https://twitter.com/scrutmydocs) > > > > Le 21 févr. 2013 à 09:35, Karel Minařík [karel.minarik@gmail.com](mailto:karel.minarik@gmail.com) a > > écrit : > > > > INPUT: story text > > > > > OUTPUT: list of symbols and scores representing the likelihood they are > > > related to this story (GOOG: 96%, MSFT: 43%, etc) > > > > > > Is there anything in elasticsearch that will help me here? > > > > Absolutely -- look at the Percolator API: > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/percolate.html). > > > > If I understand you correctly, you would register your documents as > > queries with percolator, and then ask the percolator on which of these > > queries a specific document matches. You will _not_ get scoring info > > ("GOOG: 96%") in the percolator output itself, but you can get creative > > with the registered queries, of course (using `min_score` and custom > > boosting etc). > > > > Karel > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [February 21, 2013, 9:13pm UTC](https://discuss.elastic.co/t/reverse-search-i-give-you-a-block-of-text-you-tell-me-which-indexed-documents-have-a-specific-field-value-that-matches-it/10830/11 "2013-02-21T21:13:01Z")
Interesting comments indeed. My big takeaway was "But, Lucene is raw code. Shay took Lucene, and, developed server software around that made scalable and robust. He made it well-suited for the cloud." At this point, Elasticsearch seems to be a cloud-ready scalable version of Lucene. Although there are many users doing interesting things, there is no reference implementation No Pet Store application. Given a set of tools, what kind of house can you build? There is more than just search, but the article focuses a bit too much on analytics. For those interested in NLP, there was an interesting blog post today. [http://lingpipe-blog.com/2013/02/21/want-write-oreilly-book-nlp-java/](http://lingpipe-blog.com/2013/02/21/want-write-oreilly-book-nlp-java/) Did not know that book was publicly available. Cheers, Ivan On Thu, Feb 21, 2013 at 10:48 AM, Jörg Prante [joergprante@gmail.com](mailto:joergprante@gmail.com) wrote: > Mike Volpi at [http://www.indexventures.com/\*\*blog#post/603](http://www.indexventures.com/**blog#post/603)[http://www.indexventures.com/blog#post/603](http://www.indexventures.com/blog#post/603) > > "[...] Analytics are a tricky business because ultimately, the products > end in two scenarios: (a) a skilled user to get the desired result from > them; or (b) high degrees of structure that don’t answer the really big > questions. We began to look elsewhere – what if you just wanted to “talk” > to your data – sort of like how we have become accustomed to finding what > we want as consumers from Google. [...]" > > and > > "[...] But, perhaps even more so is the vision of Elasticsearch. Through > its peaks and valleys, Big Data is here to stay. Business and consumer > experiences will never be the same because we will have vast amounts of > information to make these experiences better. But, that vision will only > come to be when we can talk to our data the way we talk to our own minds. > Elasticsearch is a huge leap ahead in that direction." > > Google-like phrases in hope to find some analytic insight? Talk to data, > like talk to own mind? Well, to be frankly, that's what natural language > processing (NLP) technologies has been designed for. > > Jörg > > Am 21.02.13 19:22, schrieb Ivan Brusic: > > > Any interesting comments? I haven't seen any. > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to elasticsearch+unsubscribe@\*\*[googlegroups.com](http://googlegroups.com)[elasticsearch%2Bunsubscribe@googlegroups.com](mailto:elasticsearch%2Bunsubscribe@googlegroups.com) > . > For more options, visit [https://groups.google.com/\*\*groups/opt\_out](https://groups.google.com/**groups/opt_out)[https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out) > . -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [February 22, 2013, 4:11pm UTC](https://discuss.elastic.co/t/reverse-search-i-give-you-a-block-of-text-you-tell-me-which-indexed-documents-have-a-specific-field-value-that-matches-it/10830/12 "2013-02-22T16:11:00Z")
Heya, The analytics part is stressed out mainly because, from my experience, when people think of elasticsearch, they mainly think of unstructured search (still). This is changing, but we want to push it out there. A company built a whole "google analytics" like product on top of elasticsearch that has no unstructured search. Its quite powerful when it comes to it, shame people won't be aware of it. Regarding NLP, or more broadly, machine learning, its definitely something that we are interested at. We really have nothing concrete there, just thoughts up in the air, and possibly some sketches here and there. Obviously its a very broad subject, we will see how we progress there. We will update obviously once things are more concrete. The nice bit about ES is that it can cover quite a bit to many people. We should do a better job at explaining all the things that can be done with it. Its definitely something that we are on... On Feb 21, 2013, at 10:13 PM, Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com) wrote: > Interesting comments indeed. > > My big takeaway was "But, Lucene is raw code. Shay took Lucene, and, developed server software around that made scalable and robust. He made it well-suited for the cloud." > > At this point, Elasticsearch seems to be a cloud-ready scalable version of Lucene. Although there are many users doing interesting things, there is no reference implementation No Pet Store application. Given a set of tools, what kind of house can you build? There is more than just search, but the article focuses a bit too much on analytics. > > For those interested in NLP, there was an interesting blog post today. [http://lingpipe-blog.com/2013/02/21/want-write-oreilly-book-nlp-java/](http://lingpipe-blog.com/2013/02/21/want-write-oreilly-book-nlp-java/) > Did not know that book was publicly available. > > Cheers, > > Ivan > > On Thu, Feb 21, 2013 at 10:48 AM, Jörg Prante [joergprante@gmail.com](mailto:joergprante@gmail.com) wrote: > Mike Volpi at [http://www.indexventures.com/blog#post/603](http://www.indexventures.com/blog#post/603) > > "[...] Analytics are a tricky business because ultimately, the products end in two scenarios: (a) a skilled user to get the desired result from them; or (b) high degrees of structure that don’t answer the really big questions. We began to look elsewhere – what if you just wanted to “talk” to your data – sort of like how we have become accustomed to finding what we want as consumers from Google. [...]" > > and > > "[...] But, perhaps even more so is the vision of Elasticsearch. Through its peaks and valleys, Big Data is here to stay. Business and consumer experiences will never be the same because we will have vast amounts of information to make these experiences better. But, that vision will only come to be when we can talk to our data the way we talk to our own minds. Elasticsearch is a huge leap ahead in that direction." > > Google-like phrases in hope to find some analytic insight? Talk to data, like talk to own mind? Well, to be frankly, that's what natural language processing (NLP) technologies has been designed for. > > Jörg > > Am 21.02.13 19:22, schrieb Ivan Brusic: > Any interesting comments? I haven't seen any. > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:50am UTC](https://discuss.elastic.co/t/reverse-search-i-give-you-a-block-of-text-you-tell-me-which-indexed-documents-have-a-specific-field-value-that-matches-it/10830/13 "2017-07-06T02:50:02Z")