# Accessing tf-idf **URL:** \ **Category:** Elasticsearch\ **Created:** [October 16, 2013, 6:34pm UTC](https://discuss.elastic.co/t/accessing-tf-idf/13976 "2013-10-16T18:34:37Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![benmccann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/benmccann/32/420_2.png) [@benmccann](https://discuss.elastic.co/u/benmccann)\ **Post date:** [October 16, 2013, 6:34pm UTC](https://discuss.elastic.co/t/accessing-tf-idf/13976/1 "2013-10-16T18:34:37Z")
Can you access the tf-idf to use outside of ElasticSearch? Also, is the tf-idf calculated on a per-field basis or a per-document basis? Thanks, Ben -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [October 16, 2013, 7:27pm UTC](https://discuss.elastic.co/t/accessing-tf-idf/13976/2 "2013-10-16T19:27:32Z")
The various elements of scoring are exposed in the explanation (if enabled). Not an ideal format to process programmatically, but the results are there. TF-IDF is calculated per-field, with the score of the document being a combination of the various TF-IDF of the fields involved. -- Ivan On Wed, Oct 16, 2013 at 11:34 AM, Ben McCann [benjamin.j.mccann@gmail.com](mailto:benjamin.j.mccann@gmail.com)wrote: > Can you access the tf-idf to use outside of Elasticsearch? Also, is the > tf-idf calculated on a per-field basis or a per-document basis? > > Thanks, > Ben > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Ben\_McCann\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ben_mccann_2/32/1097_2.png) [@Ben\_McCann\_2](https://discuss.elastic.co/u/Ben_McCann_2)\ **Post date:** [October 16, 2013, 7:38pm UTC](https://discuss.elastic.co/t/accessing-tf-idf/13976/3 "2013-10-16T19:38:14Z")
Hi Ivan, Thanks for the tip! I'm not familiar with the explanation. Is that the same as the Explain API[http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-explain.html](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-explain.html) for computing a score explanation for a query and a specific document? I'd really like to get a list of all the terms that appear in a field in my index and not only for a particular query. Thanks, Ben On Wed, Oct 16, 2013 at 12:27 PM, Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com) wrote: > The various elements of scoring are exposed in the explanation (if > enabled). Not an ideal format to process programmatically, but the results > are there. > > TF-IDF is calculated per-field, with the score of the document being a > combination of the various TF-IDF of the fields involved. > > -- > Ivan > > On Wed, Oct 16, 2013 at 11:34 AM, Ben McCann [benjamin.j.mccann@gmail.com](mailto:benjamin.j.mccann@gmail.com)wrote: > > > Can you access the tf-idf to use outside of Elasticsearch? Also, is the > > tf-idf calculated on a per-field basis or a per-document basis? > > > > Thanks, > > Ben > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/ZhDlIksA5pE/unsubscribe](https://groups.google.com/d/topic/elasticsearch/ZhDlIksA5pE/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- > **[Ben McCann on about.me](https://about.me/benmccann)** > > I am a Founder, Entrepreneur, and Investor in Mountain View, California. Visit my website. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [October 16, 2013, 7:57pm UTC](https://discuss.elastic.co/t/accessing-tf-idf/13976/4 "2013-10-16T19:57:16Z")
Not quite the explain API, but the score explanation for any query: > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. When enabled, it is similar to the Explain API, but for each document returned by the query instead of just one. TF-IDF only matters in the context of a query. If you want all the terms, you can use a term facet with a large size, or use Jorg's plugin: > **[GitHub - jprante/elasticsearch-index-termlist: Elasticsearch Index Termlist](https://github.com/jprante/elasticsearch-index-termlist)** > > Elasticsearch Index Termlist. Contribute to jprante/elasticsearch-index-termlist development by creating an account on GitHub. Cheers, Ivan On Wed, Oct 16, 2013 at 12:38 PM, Ben McCann [ben@benmccann.com](mailto:ben@benmccann.com) wrote: > Hi Ivan, > > Thanks for the tip! I'm not familiar with the explanation. Is that the > same as the Explain API[http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-explain.html](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-explain.html) for computing > a score explanation for a query and a specific document? I'd really like to > get a list of all the terms that appear in a field in my index and not only > for a particular query. > > Thanks, > Ben > > On Wed, Oct 16, 2013 at 12:27 PM, Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com) wrote: > > > The various elements of scoring are exposed in the explanation (if > > enabled). Not an ideal format to process programmatically, but the results > > are there. > > > > TF-IDF is calculated per-field, with the score of the document being a > > combination of the various TF-IDF of the fields involved. > > > > -- > > Ivan > > > > On Wed, Oct 16, 2013 at 11:34 AM, Ben McCann \<[benjamin.j.mccann@gmail.com](mailto:benjamin.j.mccann@gmail.com) > > > > > wrote: > > > > > Can you access the tf-idf to use outside of Elasticsearch? Also, is the > > > tf-idf calculated on a per-field basis or a per-document basis? > > > > > > Thanks, > > > Ben > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit > > [https://groups.google.com/d/topic/elasticsearch/ZhDlIksA5pE/unsubscribe](https://groups.google.com/d/topic/elasticsearch/ZhDlIksA5pE/unsubscribe). > > To unsubscribe from this group and all its topics, send an email to > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > [Ben McCann - Mountain View, California, Connectifier, Carnegie Mellon University | about.me](http://about.me/benmccann) > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Ben\_McCann\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ben_mccann_2/32/1097_2.png) [@Ben\_McCann\_2](https://discuss.elastic.co/u/Ben_McCann_2)\ **Post date:** [October 17, 2013, 12:30am UTC](https://discuss.elastic.co/t/accessing-tf-idf/13976/5 "2013-10-17T00:30:08Z")
Thanks Ivan! Jorg's plugin is exactly what I was looking for. I just sent him a pull request to update to 0.90.5 and he released a new version with it, so I'll give it a go and see how it works. Also, good point that tf-idf might not be exactly the right term for what I was looking for. I mainly care about the IDF portion. Thanks again! -Ben P.S. you've helped me a couple times, so just wanted to say thanks! And also I'm sure you get just as much recruiter spam as I do, so i won't bug you, but if you ever want to explore the possibility of working on elasticsearch with a bunch of ex-googlers then I'd love to share with you what we're up to in case it's interesting to you On Wed, Oct 16, 2013 at 12:57 PM, Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com) wrote: > Not quite the explain API, but the score explanation for any query: > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-explain.html) > > When enabled, it is similar to the Explain API, but for each document > returned by the query instead of just one. > > TF-IDF only matters in the context of a query. If you want all the terms, > you can use a term facet with a large size, or use Jorg's plugin: > [GitHub - jprante/elasticsearch-index-termlist: Elasticsearch Index Termlist](https://github.com/jprante/elasticsearch-index-termlist) > > Cheers, > > Ivan > > On Wed, Oct 16, 2013 at 12:38 PM, Ben McCann [ben@benmccann.com](mailto:ben@benmccann.com) wrote: > > > Hi Ivan, > > > > Thanks for the tip! I'm not familiar with the explanation. Is that the > > same as the Explain API[http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-explain.html](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-explain.html) for computing > > a score explanation for a query and a specific document? I'd really like to > > get a list of all the terms that appear in a field in my index and not only > > for a particular query. > > > > Thanks, > > Ben > > > > On Wed, Oct 16, 2013 at 12:27 PM, Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com) wrote: > > > > > The various elements of scoring are exposed in the explanation (if > > > enabled). Not an ideal format to process programmatically, but the results > > > are there. > > > > > > TF-IDF is calculated per-field, with the score of the document being a > > > combination of the various TF-IDF of the fields involved. > > > > > > -- > > > Ivan > > > > > > On Wed, Oct 16, 2013 at 11:34 AM, Ben McCann \< > > > [benjamin.j.mccann@gmail.com](mailto:benjamin.j.mccann@gmail.com)\> wrote: > > > > > > > Can you access the tf-idf to use outside of Elasticsearch? Also, is the > > > > tf-idf calculated on a per-field basis or a per-document basis? > > > > > > > > Thanks, > > > > Ben > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > -- > > > You received this message because you are subscribed to a topic in the > > > Google Groups "elasticsearch" group. > > > To unsubscribe from this topic, visit > > > [https://groups.google.com/d/topic/elasticsearch/ZhDlIksA5pE/unsubscribe](https://groups.google.com/d/topic/elasticsearch/ZhDlIksA5pE/unsubscribe). > > > To unsubscribe from this group and all its topics, send an email to > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > [Ben McCann - Mountain View, California, Connectifier, Carnegie Mellon University | about.me](http://about.me/benmccann) > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/ZhDlIksA5pE/unsubscribe](https://groups.google.com/d/topic/elasticsearch/ZhDlIksA5pE/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- > **[Ben McCann on about.me](https://about.me/benmccann)** > > I am a Founder, Entrepreneur, and Investor in Mountain View, California. Visit my website. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [October 17, 2013, 5:23am UTC](https://discuss.elastic.co/t/accessing-tf-idf/13976/6 "2013-10-17T05:23:18Z")
Thank Jorg for the plugin. The code is standard in Lucene (accessing the TermEnums), just adapted for elasticsearch. -- Ivan On Wed, Oct 16, 2013 at 5:30 PM, Ben McCann [ben@benmccann.com](mailto:ben@benmccann.com) wrote: > Thanks Ivan! Jorg's plugin is exactly what I was looking for. I just sent > him a pull request to update to 0.90.5 and he released a new version with > it, so I'll give it a go and see how it works. Also, good point that tf-idf > might not be exactly the right term for what I was looking for. I mainly > care about the IDF portion. > > Thanks again! > > -Ben > > P.S. you've helped me a couple times, so just wanted to say thanks! And > also I'm sure you get just as much recruiter spam as I do, so i won't bug > you, but if you ever want to explore the possibility of working on > elasticsearch with a bunch of ex-googlers then I'd love to share with you > what we're up to in case it's interesting to you > > On Wed, Oct 16, 2013 at 12:57 PM, Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com) wrote: > > > Not quite the explain API, but the score explanation for any query: > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-explain.html) > > > > When enabled, it is similar to the Explain API, but for each document > > returned by the query instead of just one. > > > > TF-IDF only matters in the context of a query. If you want all the > > terms, you can use a term facet with a large size, or use Jorg's plugin: > > [GitHub - jprante/elasticsearch-index-termlist: Elasticsearch Index Termlist](https://github.com/jprante/elasticsearch-index-termlist) > > > > Cheers, > > > > Ivan > > > > On Wed, Oct 16, 2013 at 12:38 PM, Ben McCann [ben@benmccann.com](mailto:ben@benmccann.com) wrote: > > > > > Hi Ivan, > > > > > > Thanks for the tip! I'm not familiar with the explanation. Is that the > > > same as the Explain API[http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-explain.html](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-explain.html) for computing > > > a score explanation for a query and a specific document? I'd really like to > > > get a list of all the terms that appear in a field in my index and not only > > > for a particular query. > > > > > > Thanks, > > > Ben > > > > > > On Wed, Oct 16, 2013 at 12:27 PM, Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com) wrote: > > > > > > > The various elements of scoring are exposed in the explanation (if > > > > enabled). Not an ideal format to process programmatically, but the results > > > > are there. > > > > > > > > TF-IDF is calculated per-field, with the score of the document being a > > > > combination of the various TF-IDF of the fields involved. > > > > > > > > -- > > > > Ivan > > > > > > > > On Wed, Oct 16, 2013 at 11:34 AM, Ben McCann \< > > > > [benjamin.j.mccann@gmail.com](mailto:benjamin.j.mccann@gmail.com)\> wrote: > > > > > > > > > Can you access the tf-idf to use outside of Elasticsearch? Also, is > > > > > the tf-idf calculated on a per-field basis or a per-document basis? > > > > > > > > > > Thanks, > > > > > Ben > > > > > > > > > > -- > > > > > You received this message because you are subscribed to the Google > > > > > Groups "elasticsearch" group. > > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > > > -- > > > > You received this message because you are subscribed to a topic in the > > > > Google Groups "elasticsearch" group. > > > > To unsubscribe from this topic, visit > > > > [https://groups.google.com/d/topic/elasticsearch/ZhDlIksA5pE/unsubscribe](https://groups.google.com/d/topic/elasticsearch/ZhDlIksA5pE/unsubscribe) > > > > . > > > > To unsubscribe from this group and all its topics, send an email to > > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > -- > > > [Ben McCann - Mountain View, California, Connectifier, Carnegie Mellon University | about.me](http://about.me/benmccann) > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit > > [https://groups.google.com/d/topic/elasticsearch/ZhDlIksA5pE/unsubscribe](https://groups.google.com/d/topic/elasticsearch/ZhDlIksA5pE/unsubscribe). > > To unsubscribe from this group and all its topics, send an email to > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > [Ben McCann - Mountain View, California, Connectifier, Carnegie Mellon University | about.me](http://about.me/benmccann) > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [October 17, 2013, 9:12am UTC](https://discuss.elastic.co/t/accessing-tf-idf/13976/7 "2013-10-17T09:12:35Z")
I love to help people getting in touch with Elasticsearch. Don't hesitate to ask questions - there are no silly questions, only silly answers. Thanks and kudos to Shay who is so dedicated and passionate and great by deciding to release so valuable distributed scalable code to the public! Jörg Am 17.10.2013 07:23 schrieb "Ivan Brusic" [ivan@brusic.com](mailto:ivan@brusic.com): > Thank Jorg for the plugin. The code is standard in Lucene (accessing the > TermEnums), just adapted for elasticsearch. > > -- > Ivan > > On Wed, Oct 16, 2013 at 5:30 PM, Ben McCann [ben@benmccann.com](mailto:ben@benmccann.com) wrote: > > > Thanks Ivan! Jorg's plugin is exactly what I was looking for. I just > > sent him a pull request to update to 0.90.5 and he released a new version > > with it, so I'll give it a go and see how it works. Also, good point that > > tf-idf might not be exactly the right term for what I was looking for. I > > mainly care about the IDF portion. > > > > Thanks again! > > > > -Ben > > > > P.S. you've helped me a couple times, so just wanted to say thanks! And > > also I'm sure you get just as much recruiter spam as I do, so i won't bug > > you, but if you ever want to explore the possibility of working on > > elasticsearch with a bunch of ex-googlers then I'd love to share with you > > what we're up to in case it's interesting to you > > > > On Wed, Oct 16, 2013 at 12:57 PM, Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com) wrote: > > > > > Not quite the explain API, but the score explanation for any query: > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-explain.html) > > > > > > When enabled, it is similar to the Explain API, but for each document > > > returned by the query instead of just one. > > > > > > TF-IDF only matters in the context of a query. If you want all the > > > terms, you can use a term facet with a large size, or use Jorg's plugin: > > > [GitHub - jprante/elasticsearch-index-termlist: Elasticsearch Index Termlist](https://github.com/jprante/elasticsearch-index-termlist) > > > > > > Cheers, > > > > > > Ivan > > > > > > On Wed, Oct 16, 2013 at 12:38 PM, Ben McCann [ben@benmccann.com](mailto:ben@benmccann.com) wrote: > > > > > > > Hi Ivan, > > > > > > > > Thanks for the tip! I'm not familiar with the explanation. Is that the > > > > same as the Explain API[http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-explain.html](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-explain.html) for computing > > > > a score explanation for a query and a specific document? I'd really like to > > > > get a list of all the terms that appear in a field in my index and not only > > > > for a particular query. > > > > > > > > Thanks, > > > > Ben > > > > > > > > On Wed, Oct 16, 2013 at 12:27 PM, Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com) wrote: > > > > > > > > > The various elements of scoring are exposed in the explanation (if > > > > > enabled). Not an ideal format to process programmatically, but the results > > > > > are there. > > > > > > > > > > TF-IDF is calculated per-field, with the score of the document being a > > > > > combination of the various TF-IDF of the fields involved. > > > > > > > > > > -- > > > > > Ivan > > > > > > > > > > On Wed, Oct 16, 2013 at 11:34 AM, Ben McCann \< > > > > > [benjamin.j.mccann@gmail.com](mailto:benjamin.j.mccann@gmail.com)\> wrote: > > > > > > > > > > > Can you access the tf-idf to use outside of Elasticsearch? Also, is > > > > > > the tf-idf calculated on a per-field basis or a per-document basis? > > > > > > > > > > > > Thanks, > > > > > > Ben > > > > > > > > > > > > -- > > > > > > You received this message because you are subscribed to the Google > > > > > > Groups "elasticsearch" group. > > > > > > To unsubscribe from this group and stop receiving emails from it, > > > > > > send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > > > > > -- > > > > > You received this message because you are subscribed to a topic in the > > > > > Google Groups "elasticsearch" group. > > > > > To unsubscribe from this topic, visit > > > > > [https://groups.google.com/d/topic/elasticsearch/ZhDlIksA5pE/unsubscribe](https://groups.google.com/d/topic/elasticsearch/ZhDlIksA5pE/unsubscribe) > > > > > . > > > > > To unsubscribe from this group and all its topics, send an email to > > > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > > > -- > > > > [Ben McCann - Mountain View, California, Connectifier, Carnegie Mellon University | about.me](http://about.me/benmccann) > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > -- > > > You received this message because you are subscribed to a topic in the > > > Google Groups "elasticsearch" group. > > > To unsubscribe from this topic, visit > > > [https://groups.google.com/d/topic/elasticsearch/ZhDlIksA5pE/unsubscribe](https://groups.google.com/d/topic/elasticsearch/ZhDlIksA5pE/unsubscribe). > > > To unsubscribe from this group and all its topics, send an email to > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > [Ben McCann - Mountain View, California, Connectifier, Carnegie Mellon University | about.me](http://about.me/benmccann) > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Ramdev\_Wudali](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ramdev_wudali/32/7894_2.png) [@Ramdev\_Wudali](https://discuss.elastic.co/u/Ramdev_Wudali)\ **Post date:** [April 25, 2014, 3:46pm UTC](https://discuss.elastic.co/t/accessing-tf-idf/13976/8 "2014-04-25T15:46:53Z")
A variant on this particular request: I would like to get the tf-idf for an indexed field. (the field is a body of a news document). I would like to find discriminating terms in the document set (the document set is a result of executing a filter on the search index. The discriminating terms are to help with improving the query as the number of documents returned are too many and relevant documents are getting lost in the search result (of executing a filter). Is it possible to run the tf-idf calculations that Elastic does while indexing the document.(the API to access the TF-IDF calculations) Thanks Ramdev On Thursday, 17 October 2013 04:12:35 UTC-5, Jörg Prante wrote: > I love to help people getting in touch with Elasticsearch. Don't hesitate > to ask questions - there are no silly questions, only silly answers. > > Thanks and kudos to Shay who is so dedicated and passionate and great by > deciding to release so valuable distributed scalable code to the public! > > Jörg > Am 17.10.2013 07:23 schrieb "Ivan Brusic" \<[iv...@brusic.com](mailto:iv...@brusic.com) \ > > > : > > > Thank Jorg for the plugin. The code is standard in Lucene (accessing the > > TermEnums), just adapted for elasticsearch. > > > > -- > > Ivan > > > > On Wed, Oct 16, 2013 at 5:30 PM, Ben McCann \<[b...@benmccann.com](mailto:b...@benmccann.com)\ > > > > > wrote: > > > > > Thanks Ivan! Jorg's plugin is exactly what I was looking for. I just > > > sent him a pull request to update to 0.90.5 and he released a new version > > > with it, so I'll give it a go and see how it works. Also, good point that > > > tf-idf might not be exactly the right term for what I was looking for. I > > > mainly care about the IDF portion. > > > > > > Thanks again! > > > > > > -Ben > > > > > > P.S. you've helped me a couple times, so just wanted to say thanks! And > > > also I'm sure you get just as much recruiter spam as I do, so i won't bug > > > you, but if you ever want to explore the possibility of working on > > > elasticsearch with a bunch of ex-googlers then I'd love to share with you > > > what we're up to in case it's interesting to you > > > > > > On Wed, Oct 16, 2013 at 12:57 PM, Ivan Brusic \<[iv...@brusic.com](mailto:iv...@brusic.com)\ > > > > > > > wrote: > > > > > > > Not quite the explain API, but the score explanation for any query: > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-explain.html) > > > > > > > > When enabled, it is similar to the Explain API, but for each document > > > > returned by the query instead of just one. > > > > > > > > TF-IDF only matters in the context of a query. If you want all the > > > > terms, you can use a term facet with a large size, or use Jorg's plugin: > > > > [GitHub - jprante/elasticsearch-index-termlist: Elasticsearch Index Termlist](https://github.com/jprante/elasticsearch-index-termlist) > > > > > > > > Cheers, > > > > > > > > Ivan > > > > > > > > On Wed, Oct 16, 2013 at 12:38 PM, Ben McCann \<[b...@benmccann.com](mailto:b...@benmccann.com)\ > > > > > > > > > wrote: > > > > > > > > > Hi Ivan, > > > > > > > > > > Thanks for the tip! I'm not familiar with the explanation. Is that > > > > > the same as the Explain API[http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-explain.html](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-explain.html) for computing > > > > > a score explanation for a query and a specific document? I'd really like to > > > > > get a list of all the terms that appear in a field in my index and not only > > > > > for a particular query. > > > > > > > > > > Thanks, > > > > > Ben > > > > > > > > > > On Wed, Oct 16, 2013 at 12:27 PM, Ivan Brusic \<[iv...@brusic.com](mailto:iv...@brusic.com)\ > > > > > > > > > > > wrote: > > > > > > > > > > > The various elements of scoring are exposed in the explanation (if > > > > > > enabled). Not an ideal format to process programmatically, but the results > > > > > > are there. > > > > > > > > > > > > TF-IDF is calculated per-field, with the score of the document being > > > > > > a combination of the various TF-IDF of the fields involved. > > > > > > > > > > > > -- > > > > > > Ivan > > > > > > > > > > > > On Wed, Oct 16, 2013 at 11:34 AM, Ben McCann \<[benjamin...@gmail.com](mailto:benjamin...@gmail.com)\ > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > Can you access the tf-idf to use outside of Elasticsearch? Also, is > > > > > > > the tf-idf calculated on a per-field basis or a per-document basis? > > > > > > > > > > > > > > Thanks, > > > > > > > Ben > > > > > > > > > > > > > > -- > > > > > > > You received this message because you are subscribed to the Google > > > > > > > Groups "elasticsearch" group. > > > > > > > To unsubscribe from this group and stop receiving emails from it, > > > > > > > send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > > > > > > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > > > > > > > -- > > > > > > You received this message because you are subscribed to a topic in > > > > > > the Google Groups "elasticsearch" group. > > > > > > To unsubscribe from this topic, visit > > > > > > [https://groups.google.com/d/topic/elasticsearch/ZhDlIksA5pE/unsubscribe](https://groups.google.com/d/topic/elasticsearch/ZhDlIksA5pE/unsubscribe) > > > > > > . > > > > > > To unsubscribe from this group and all its topics, send an email to > > > > > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > > > > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > > > > > -- > > > > > [Ben McCann - Mountain View, California, Connectifier, Carnegie Mellon University | about.me](http://about.me/benmccann) > > > > > > > > > > -- > > > > > You received this message because you are subscribed to the Google > > > > > Groups "elasticsearch" group. > > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > > > -- > > > > You received this message because you are subscribed to a topic in the > > > > Google Groups "elasticsearch" group. > > > > To unsubscribe from this topic, visit > > > > [https://groups.google.com/d/topic/elasticsearch/ZhDlIksA5pE/unsubscribe](https://groups.google.com/d/topic/elasticsearch/ZhDlIksA5pE/unsubscribe) > > > > . > > > > To unsubscribe from this group and all its topics, send an email to > > > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > -- > > > [Ben McCann - Mountain View, California, Connectifier, Carnegie Mellon University | about.me](http://about.me/benmccann) > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/fd6c72a7-9f3f-4764-b512-9234a6769958%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/fd6c72a7-9f3f-4764-b512-9234a6769958%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [April 25, 2014, 6:09pm UTC](https://discuss.elastic.co/t/accessing-tf-idf/13976/9 "2014-04-25T18:09:35Z")
Can you provide a small example of what you are trying to achieve? Are the discriminating terms known beforehand or is it dependent on the document? Have you looked into the new text scoring features which have been released since the original post? It is worth looking into: > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. You can probably calculate the TF values during indexing, but not the IDF since that value is based on all of the documents in a shard. Cheers, Ivan On Fri, Apr 25, 2014 at 8:46 AM, Ramdev Wudali [agastya71@gmail.com](mailto:agastya71@gmail.com) wrote: > A variant on this particular request: > > I would like to get the tf-idf for an indexed field. (the field is a body > of a news document). I would like to find discriminating terms in the > document set (the document set is a result of executing a filter on the > search index. > The discriminating terms are to help with improving the query as the > number of documents returned are too many and relevant documents are > getting lost in the search result (of executing a filter). > > Is it possible to run the tf-idf calculations that Elastic does while > indexing the document.(the API to access the TF-IDF calculations) > > Thanks > > Ramdev -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQAhqGSS8L5sEGe-d%2BwYmwC-fiMZP0FhyDa\_UCf9xB8GGA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQAhqGSS8L5sEGe-d%2BwYmwC-fiMZP0FhyDa_UCf9xB8GGA%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Ramdev\_Wudali](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ramdev_wudali/32/7894_2.png) [@Ramdev\_Wudali](https://discuss.elastic.co/u/Ramdev_Wudali)\ **Post date:** [April 28, 2014, 1:48pm UTC](https://discuss.elastic.co/t/accessing-tf-idf/13976/10 "2014-04-28T13:48:11Z")
Ivan: I filter the index for documents containing AAPL(the ticket symbol) (as part of a field that is filterable). I get back 1000 documents in no particular order as the request was just a filter. To this filter, I would like to add a "discriminating/significant" text that would be found in the 1000 documents. So that the documents returned are in a sense only those that are significant. I do not want the terms to be significant against the whole index, but only against the documents that are returned for the query. Hence I would like to run some extra analysis against this filter request result to identify these "discriminating/significant" terms. I was wondering if I can access the elastic API /underlying implementation to do the calculations. Ramdev On Friday, 25 April 2014 13:09:35 UTC-5, Ivan Brusic wrote: > Can you provide a small example of what you are trying to achieve? Are the > discriminating terms known beforehand or is it dependent on the document? > Have you looked into the new text scoring features which have been released > since the original post? It is worth looking into: > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-advanced-scripting.html) > > You can probably calculate the TF values during indexing, but not the IDF > since that value is based on all of the documents in a shard. > > Cheers, > > Ivan > > On Fri, Apr 25, 2014 at 8:46 AM, Ramdev Wudali \<[agas...@gmail.com](mailto:agas...@gmail.com)\ > > > wrote: > > > A variant on this particular request: > > > > I would like to get the tf-idf for an indexed field. (the field is a body > > of a news document). I would like to find discriminating terms in the > > document set (the document set is a result of executing a filter on the > > search index. > > The discriminating terms are to help with improving the query as the > > number of documents returned are too many and relevant documents are > > getting lost in the search result (of executing a filter). > > > > Is it possible to run the tf-idf calculations that Elastic does while > > indexing the document.(the API to access the TF-IDF calculations) > > > > Thanks > > > > Ramdev -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/81a1726e-3b08-4de8-b9ea-28b159516e40%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/81a1726e-3b08-4de8-b9ea-28b159516e40%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [April 28, 2014, 7:06pm UTC](https://discuss.elastic.co/t/accessing-tf-idf/13976/11 "2014-04-28T19:06:55Z")
Would it be possible to create some sort of numerical value from the discriminating/significant text at index time in order to sort the documents by? You can index the documents with term vectors, which will allow you to access the term frequency values: > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. Not sure if those values can be used in script or even to sort by. Using scripts, you can get access to the fields. It would be time-consuming, but you can iterate through each term of a field and use the text scoring features to get the appropriate values. Cheers, Ivan On Mon, Apr 28, 2014 at 6:48 AM, Ramdev Wudali [agastya71@gmail.com](mailto:agastya71@gmail.com) wrote: > Ivan: > I filter the index for documents containing AAPL(the ticket symbol) (as > part of a field that is filterable). > I get back 1000 documents in no particular order as the request was just a > filter. To this filter, I would like to add a "discriminating/significant" > text that would be found in the 1000 documents. So that the documents > returned are in a sense only those that are significant. > > I do not want the terms to be significant against the whole index, but > only against the documents that are returned for the query. Hence I would > like to run some extra analysis against this filter request result to > identify these "discriminating/significant" terms. > > I was wondering if I can access the elastic API /underlying implementation > to do the calculations. > > Ramdev > > On Friday, 25 April 2014 13:09:35 UTC-5, Ivan Brusic wrote: > > > Can you provide a small example of what you are trying to achieve? Are > > the discriminating terms known beforehand or is it dependent on the > > document? Have you looked into the new text scoring features which have > > been released since the original post? It is worth looking into: > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/) > > reference/current/modules-advanced-scripting.html > > > > You can probably calculate the TF values during indexing, but not the IDF > > since that value is based on all of the documents in a shard. > > > > Cheers, > > > > Ivan > > > > On Fri, Apr 25, 2014 at 8:46 AM, Ramdev Wudali [agas...@gmail.com](mailto:agas...@gmail.com) wrote: > > > > > A variant on this particular request: > > > > > > I would like to get the tf-idf for an indexed field. (the field is a > > > body of a news document). I would like to find discriminating terms in the > > > document set (the document set is a result of executing a filter on the > > > search index. > > > The discriminating terms are to help with improving the query as the > > > number of documents returned are too many and relevant documents are > > > getting lost in the search result (of executing a filter). > > > > > > Is it possible to run the tf-idf calculations that Elastic does while > > > indexing the document.(the API to access the TF-IDF calculations) > > > > > > Thanks > > > > > > Ramdev > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/81a1726e-3b08-4de8-b9ea-28b159516e40%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/81a1726e-3b08-4de8-b9ea-28b159516e40%40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/81a1726e-3b08-4de8-b9ea-28b159516e40%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/81a1726e-3b08-4de8-b9ea-28b159516e40%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDgHyiEDcs1zLdAMqVuQV6SO9nOk9SZHNLSyXjC3tHDSQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDgHyiEDcs1zLdAMqVuQV6SO9nOk9SZHNLSyXjC3tHDSQ%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Vladi\_Feigin](https://avatars.discourse-cdn.com/v4/letter/v/258eb7/32.png) [@Vladi\_Feigin](https://discuss.elastic.co/u/Vladi_Feigin)\ **Post date:** [May 5, 2014, 1:34pm UTC](https://discuss.elastic.co/t/accessing-tf-idf/13976/12 "2014-05-05T13:34:26Z")
Hello, I have a requirement to retrieve the terms frequency (TF) from all recently indexed documents (last 24 hours) So in a query I have to supply the time range and expect to get the TFs of all terms in the given time range Is it possible to do in ES? If yes, please refer me to the documentation. Thank you in advance, Vlad On Wednesday, October 16, 2013 9:34:37 PM UTC+3, Ben McCann wrote: > Can you access the tf-idf to use outside of Elasticsearch? Also, is the > tf-idf calculated on a per-field basis or a per-document basis? > > Thanks, > Ben -- This message may contain confidential and/or privileged information. If you are not the addressee or authorized to receive this on behalf of the addressee you must not use, copy, disclose or take action based on this message or any information herein. If you have received this message in error, please advise the sender immediately by reply email and delete this message. Thank you. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/c0e230bc-e7db-45d4-a263-54e10467a696%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/c0e230bc-e7db-45d4-a263-54e10467a696%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:31am UTC](https://discuss.elastic.co/t/accessing-tf-idf/13976/13 "2017-07-06T01:31:40Z")