# Highlighting on Fields that aren't Stored or in \_Source **URL:** \ **Category:** Elasticsearch\ **Created:** [April 17, 2012, 9:52pm UTC](https://discuss.elastic.co/t/highlighting-on-fields-that-arent-stored-or-in--source/7371 "2012-04-17T21:52:56Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![John\_Bartak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/john_bartak/32/1830_2.png) [@John\_Bartak](https://discuss.elastic.co/u/John_Bartak)\ **Post date:** [April 17, 2012, 9:52pm UTC](https://discuss.elastic.co/t/highlighting-on-fields-that-arent-stored-or-in--source/7371/1 "2012-04-17T21:52:56Z")
Here's my situation: I'm planning an ElasticSearch solution that will index millions (possibly hundreds of millions) of documents that will run on EC2 instance-store instances. The amount of storage you get per instances is limited (1.6 TB for X-Large instances) so I was planning on storing as little data as possible in the index. But I want to be able to highlight fields that aren't stored in the index. I'm planning on using Tika (and a few other tools) to extract content and I'll store the extracted content in S3 (so I don't have to re-extract the content every time I re-index a document). So is there some way I can highlight content that is in S3, probably as a post-processing step of the search? Looking at the source code, it seems like it should be possible to do something similar to what was done with the SourceSimpleFragmentsBuilder. Basically inherit from SimpleFramentsBuilder and override createFragments so if there are no values return a JSON document that contains the startOffset, endOffset, and termOffsets for each highlight. Then have my code use those offsets with the content stored in S3 to build highlights. (Of course this only works for fields that store term-vectors). Does this seem like a reasonable idea or would I just be better off storing compressed data into the \_Source field? ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [April 19, 2012, 1:39pm UTC](https://discuss.elastic.co/t/highlighting-on-fields-that-arent-stored-or-in--source/7371/2 "2012-04-19T13:39:49Z")
There is and issue open to allow to get the \_source of a document from an external storage (which can be implemented as a plugin). Its not there yet though... On Wed, Apr 18, 2012 at 12:52 AM, JBartak [john.bartak@autodesk.com](mailto:john.bartak@autodesk.com) wrote: > Here's my situation: I'm planning an Elasticsearch solution that will > index millions (possibly hundreds of millions) of documents that will run > on EC2 instance-store instances. The amount of storage you get per > instances is limited (1.6 TB for X-Large instances) so I was planning on > storing as little data as possible in the index. But I want to be able to > highlight fields that aren't stored in the index. I'm planning on using > Tika (and a few other tools) to extract content and I'll store the > extracted content in S3 (so I don't have to re-extract the content every > time I re-index a document). > > So is there some way I can highlight content that is in S3, probably as a > post-processing step of the search? > > Looking at the source code, it seems like it should be possible to do > something similar to what was done with the SourceSimpleFragmentsBuilder. > Basically inherit from SimpleFramentsBuilder and override createFragments > so if there are no values return a JSON document that contains the > startOffset, endOffset, and termOffsets for each highlight. Then have my > code use those offsets with the content stored in S3 to build highlights. > (Of course this only works for fields that store term-vectors). > > Does this seem like a reasonable idea or would I just be better off > storing compressed data into the \_Source field? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:31am UTC](https://discuss.elastic.co/t/highlighting-on-fields-that-arent-stored-or-in--source/7371/3 "2017-07-06T03:31:58Z")