# Calculating a "computed" value based on index statistics / term frequencies **URL:** \ **Category:** Elasticsearch\ **Created:** [March 4, 2014, 9:56am UTC](https://discuss.elastic.co/t/calculating-a-computed-value-based-on-index-statistics-term-frequencies/16136 "2014-03-04T09:56:04Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![Kevin\_Blaisdell](https://avatars.discourse-cdn.com/v4/letter/k/e9c0ed/32.png) [@Kevin\_Blaisdell](https://discuss.elastic.co/u/Kevin_Blaisdell)\ **Post date:** [March 4, 2014, 9:56am UTC](https://discuss.elastic.co/t/calculating-a-computed-value-based-on-index-statistics-term-frequencies/16136/1 "2014-03-04T09:56:04Z")
As background I have some Lucene based code which is used to manipulate index statistics to generate numeric document vectors. This code sits between systems that need document vectors for input and Lucene indexes that are the store of the source data & statistics (term & document frequencies). I not too familiar with ES and hoping I could get some pointers to understand where best to focus my investigation or if am taking a totally incompatible approach. I am just starting to try dig into how the plugin framework works. Is a separate plugin that could be called as extensions to the native API a logical place to do this? Assuming a plugin can expose native API and not just a REST extension. Is there an approach in ES that allows you to have query results include a "computed" field instead of creating an entirely separate plugin & API extension for this? What I mean here is a field that would be the result of some custom code running on query instead of from an actual stored field in the index. Finally, is there a way to inject some preprocessing into the ES indexing / ingestion pipeline where you could precompute and store calculated fields? I believe SOLR has something for this and ES doesn't because of the potential to slow down indexing, but perhaps I am just not looking in the right place. Thanks Kevin -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/028b85e6-a14e-4512-8dff-0f4da6253d29%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/028b85e6-a14e-4512-8dff-0f4da6253d29%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Kevin\_Blaisdell](https://avatars.discourse-cdn.com/v4/letter/k/e9c0ed/32.png) [@Kevin\_Blaisdell](https://discuss.elastic.co/u/Kevin_Blaisdell)\ **Post date:** [March 4, 2014, 10:15am UTC](https://discuss.elastic.co/t/calculating-a-computed-value-based-on-index-statistics-term-frequencies/16136/2 "2014-03-04T10:15:18Z")
Quick correction. I remembered precomputing prior to population of the index wouldn't work for me in this case because there wouldn't be the term frequency data for the full corpus. On Tuesday, March 4, 2014 11:56:04 AM UTC+2, Kevin B wrote: > As background I have some Lucene based code which is used to manipulate > index statistics to generate numeric document vectors. This code sits > between systems that need document vectors for input and Lucene indexes > that are the store of the source data & statistics (term & document > frequencies). I not too familiar with ES and hoping I could get some > pointers to understand where best to focus my investigation or if am taking > a totally incompatible approach. > > I am just starting to try dig into how the plugin framework works. Is a > separate plugin that could be called as extensions to the native API a > logical place to do this? Assuming a plugin can expose native API and not > just a REST extension. > > Is there an approach in ES that allows you to have query results include a > "computed" field instead of creating an entirely separate plugin & API > extension for this? What I mean here is a field that would be the result > of some custom code running on query instead of from an actual stored field > in the index. > > Finally, is there a way to inject some preprocessing into the ES indexing > / ingestion pipeline where you could precompute and store calculated > fields? I believe SOLR has something for this and ES doesn't because of > the potential to slow down indexing, but perhaps I am just not looking in > the right place. > > Thanks > Kevin -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/f56b2437-e007-4770-80b7-16e4401524d4%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f56b2437-e007-4770-80b7-16e4401524d4%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:45am UTC](https://discuss.elastic.co/t/calculating-a-computed-value-based-on-index-statistics-term-frequencies/16136/3 "2017-07-06T01:45:52Z")