# Ann: Elasticsearch Index Termlist Plugin **URL:** \ **Category:** Elasticsearch\ **Created:** [March 25, 2012, 11:16am UTC](https://discuss.elastic.co/t/ann-elasticsearch-index-termlist-plugin/7115 "2012-03-25T11:16:56Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [March 25, 2012, 11:16am UTC](https://discuss.elastic.co/t/ann-elasticsearch-index-termlist-plugin/7115/1 "2012-03-25T11:16:56Z")
Hi, almost forgot to release a new plugin, the Termlist plugin, but here it is. You can find it here: > **[jprante/elasticsearch-index-termlist](https://github.com/jprante/elasticsearch-index-termlist)** > > elasticsearch-index-termlist - Elasticsearch Index Termlist This plugin extends Elasticsearch with a term list capability. Term lists can be generated from indexes, or even of all of the indexes in the cluster. Getting the list of all terms indexed is useful for variuos purposes, for example - building dictionaries - controlling the overall effects of analyzers on the indexed terms - automatic query building on indexed terms, e.g. for load tests - input to linguistic analysis tools - for other post-processing of the indexed terms outside of Elasticsearch Example of getting the term list of index `test` curl -XPUT '[http://localhost:9200/test/](http://localhost:9200/test/)' curl -XPUT '[http://localhost:9200/test/test/1](http://localhost:9200/test/test/1)' -d '{ "test": "Hello World" }' curl -XPUT '[http://localhost:9200/test/test/2](http://localhost:9200/test/test/2)' -d '{ "test": "Hello Jörg Prante" }' curl -XPUT '[http://localhost:9200/test/test/3](http://localhost:9200/test/test/3)' -d '{ "message": "elastic search" }' curl -XGET '[http://localhost:9200/test/\_termlist](http://localhost:9200/test/_termlist)' {"ok":true,"\_shards":{"total":5,"successful":5,"failed":0},"terms":["hello","prant","world","elastic","search","jorg"]} Have a nice weekend, Jörg ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [March 25, 2012, 12:49pm UTC](https://discuss.elastic.co/t/ann-elasticsearch-index-termlist-plugin/7115/2 "2012-03-25T12:49:00Z")
Heya, Had a very quick look at the code, you can register a custom action by having the plugin be called with ActionModule (on the plugin, with the onModule method), and then register the action on the ActionModule itself. On Sun, Mar 25, 2012 at 1:16 PM, Jörg Prante [joergprante@gmail.com](mailto:joergprante@gmail.com) wrote: > Hi, > > almost forgot to release a new plugin, the Termlist plugin, but here it > is. You can find it here: > > [GitHub - jprante/elasticsearch-index-termlist: Elasticsearch Index Termlist](https://github.com/jprante/elasticsearch-index-termlist) > > This plugin extends Elasticsearch with a term list capability. Term lists > can be generated from indexes, or even of all of the indexes in the cluster. > > Getting the list of all terms indexed is useful for variuos purposes, for > example > > - building dictionaries > - controlling the overall effects of analyzers on the indexed terms > - automatic query building on indexed terms, e.g. for load tests > - input to linguistic analysis tools > - for other post-processing of the indexed terms outside of Elasticsearch > > Example of getting the term list of index `test` > > curl -XPUT '[http://localhost:9200/test/](http://localhost:9200/test/)' > curl -XPUT '[http://localhost:9200/test/test/1](http://localhost:9200/test/test/1)' -d '{ "test": "Hello > World" }' > curl -XPUT '[http://localhost:9200/test/test/2](http://localhost:9200/test/test/2)' -d '{ "test": "Hello Jörg > Prante" }' > curl -XPUT '[http://localhost:9200/test/test/3](http://localhost:9200/test/test/3)' -d '{ "message": "elastic > search" }' > curl -XGET '[http://localhost:9200/test/\_termlist](http://localhost:9200/test/_termlist)' > > {"ok":true,"\_shards":{"total":5,"successful":5,"failed":0},"terms":["hello","prant","world","elastic","search","jorg"]} > > Have a nice weekend, > > Jörg ---
**Author:** ![vineeth\_mohan](https://avatars.discourse-cdn.com/v4/letter/v/bc79bd/32.png) [@vineeth\_mohan](https://discuss.elastic.co/u/vineeth_mohan)\ **Post date:** [March 25, 2012, 4:53pm UTC](https://discuss.elastic.co/t/ann-elasticsearch-index-termlist-plugin/7115/3 "2012-03-25T16:53:32Z")
Can we use this plugin in the following way - I want to see what all words are present in the matched documents for a particular query. - Also i want to know the frequency of occurrence of the word. Like i want to know the hot topic on news between date X and Y. I can run a range query , see the occurrence of the words and see which is the word that come the most. Thanks Vineeth On Sun, Mar 25, 2012 at 4:46 PM, Jörg Prante [joergprante@gmail.com](mailto:joergprante@gmail.com) wrote: > Hi, > > almost forgot to release a new plugin, the Termlist plugin, but here it > is. You can find it here: > > [GitHub - jprante/elasticsearch-index-termlist: Elasticsearch Index Termlist](https://github.com/jprante/elasticsearch-index-termlist) > > This plugin extends Elasticsearch with a term list capability. Term lists > can be generated from indexes, or even of all of the indexes in the cluster. > > Getting the list of all terms indexed is useful for variuos purposes, for > example > > - building dictionaries > - controlling the overall effects of analyzers on the indexed terms > - automatic query building on indexed terms, e.g. for load tests > - input to linguistic analysis tools > - for other post-processing of the indexed terms outside of Elasticsearch > > Example of getting the term list of index `test` > > curl -XPUT '[http://localhost:9200/test/](http://localhost:9200/test/)' > curl -XPUT '[http://localhost:9200/test/test/1](http://localhost:9200/test/test/1)' -d '{ "test": "Hello > World" }' > curl -XPUT '[http://localhost:9200/test/test/2](http://localhost:9200/test/test/2)' -d '{ "test": "Hello Jörg > Prante" }' > curl -XPUT '[http://localhost:9200/test/test/3](http://localhost:9200/test/test/3)' -d '{ "message": "elastic > search" }' > curl -XGET '[http://localhost:9200/test/\_termlist](http://localhost:9200/test/_termlist)' > > {"ok":true,"\_shards":{"total":5,"successful":5,"failed":0},"terms":["hello","prant","world","elastic","search","jorg"]} > > Have a nice weekend, > > Jörg ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [March 26, 2012, 6:52pm UTC](https://discuss.elastic.co/t/ann-elasticsearch-index-termlist-plugin/7115/4 "2012-03-26T18:52:57Z")
Thanks Shay, I digged deeper and I found out that something like public void onModule(ActionModule module) { module.registerAction(TermlistAction.INSTANCE, TransportTermlistAction.class); } in the plugin class seems all I need to do, indeed. Jörg On Sunday, March 25, 2012 2:49:00 PM UTC+2, kimchy wrote: > Heya, > > Had a very quick look at the code, you can register a custom action by > having the plugin be called with ActionModule (on the plugin, with the > onModule method), and then register the action on the ActionModule itself. > > On Sun, Mar 25, 2012 at 1:16 PM, Jörg Prante [joergprante@gmail.com](mailto:joergprante@gmail.com)wrote: > > > Hi, > > > > almost forgot to release a new plugin, the Termlist plugin, but here it > > is. You can find it here: > > > > [GitHub - jprante/elasticsearch-index-termlist: Elasticsearch Index Termlist](https://github.com/jprante/elasticsearch-index-termlist) > > > > This plugin extends Elasticsearch with a term list capability. Term lists > > can be generated from indexes, or even of all of the indexes in the cluster. > > > > Getting the list of all terms indexed is useful for variuos purposes, for > > example > > > > - building dictionaries > > - controlling the overall effects of analyzers on the indexed terms > > - automatic query building on indexed terms, e.g. for load tests > > - input to linguistic analysis tools > > - for other post-processing of the indexed terms outside of Elasticsearch > > > > Example of getting the term list of index `test` > > > > curl -XPUT '[http://localhost:9200/test/](http://localhost:9200/test/)' > > curl -XPUT '[http://localhost:9200/test/test/1](http://localhost:9200/test/test/1)' -d '{ "test": "Hello > > World" }' > > curl -XPUT '[http://localhost:9200/test/test/2](http://localhost:9200/test/test/2)' -d '{ "test": "Hello Jörg > > Prante" }' > > curl -XPUT '[http://localhost:9200/test/test/3](http://localhost:9200/test/test/3)' -d '{ "message": "elastic > > search" }' > > curl -XGET '[http://localhost:9200/test/\_termlist](http://localhost:9200/test/_termlist)' > > > > {"ok":true,"\_shards":{"total":5,"successful":5,"failed":0},"terms":["hello","prant","world","elastic","search","jorg"]} > > > > Have a nice weekend, > > > > Jörg ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [March 26, 2012, 7:15pm UTC](https://discuss.elastic.co/t/ann-elasticsearch-index-termlist-plugin/7115/5 "2012-03-26T19:15:48Z")
Hi, On Sunday, March 25, 2012 6:53:32 PM UTC+2, Vineeth Mohan wrote: > Can we use this plugin in the following way > > - I want to see what all words are present in the matched documents > for a particular query. Unfortunately, dumping term lists is far from being able to evaluate queries and trace all the participating fields. But I added a field confinement. In version 1.1.0, it is possible to dump terms from a given field in a given index. > - Also i want to know the frequency of occurrence of the word. > > The frequency available in the Lucene IndexReader is the "document > frequency" of a term. This looks like a very expensive call for each > term. I'm hesitating to implement it. See also the javadoc of > org.apache.lucene.search.similar.MoreLikeThis [MoreLikeThis (Lucene 3.5.0 API)](http://lucene.apache.org/core/old_versioned_docs/versions/3_5_0/api/contrib-queries/org/apache/lucene/search/similar/MoreLikeThis.html) > Like i want to know the hot topic on news between date X and Y. > I can run a range query , see the occurrence of the words and see which is > the word that come the most. If index names or field names can be organized to represent time periods, the term list dump could be used to dump all terms that are of interest. Have you considered the percolator feature? Looks like the percolator is also close to the "hot topics" feature. Jörg ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [March 26, 2012, 7:42pm UTC](https://discuss.elastic.co/t/ann-elasticsearch-index-termlist-plugin/7115/6 "2012-03-26T19:42:41Z")
Just another hint. With Lucene 4.0, it will be possible to get the total term frequency, see [https://issues.apache.org/jira/browse/LUCENE-2862](https://issues.apache.org/jira/browse/LUCENE-2862) So, let's look forward 🙂 Jörg On Monday, March 26, 2012 9:15:48 PM UTC+2, Jörg Prante wrote: > Hi, > > On Sunday, March 25, 2012 6:53:32 PM UTC+2, Vineeth Mohan wrote: > > > Can we use this plugin in the following way > > > > - I want to see what all words are present in the matched documents > > for a particular query. > > Unfortunately, dumping term lists is far from being able to evaluate > queries and trace all the participating fields. But I added a field > confinement. In version 1.1.0, it is possible to dump terms from a given > field in a given index. > > > - Also i want to know the frequency of occurrence of the word. > > > > The frequency available in the Lucene IndexReader is the "document > > frequency" of a term. This looks like a very expensive call for each > > term. I'm hesitating to implement it. See also the javadoc of > > org.apache.lucene.search.similar.MoreLikeThis > > [MoreLikeThis (Lucene 3.5.0 API)](http://lucene.apache.org/core/old_versioned_docs/versions/3_5_0/api/contrib-queries/org/apache/lucene/search/similar/MoreLikeThis.html) > > > Like i want to know the hot topic on news between date X and Y. > > I can run a range query , see the occurrence of the words and see which > > is the word that come the most. > > If index names or field names can be organized to represent time periods, > the term list dump could be used to dump all terms that are of > interest. Have you considered the percolator feature? Looks like the > percolator is also close to the "hot topics" feature. > > Jörg ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:34am UTC](https://discuss.elastic.co/t/ann-elasticsearch-index-termlist-plugin/7115/7 "2017-07-06T03:34:42Z")