# Howto: Access Character Offset of term in string field **URL:** \ **Category:** Elasticsearch\ **Created:** [May 10, 2013, 2:12pm UTC](https://discuss.elastic.co/t/howto-access-character-offset-of-term-in-string-field/11914 "2013-05-10T14:12:27Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Max\_Hoffmann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/max_hoffmann/32/2356_2.png) [@Max\_Hoffmann](https://discuss.elastic.co/u/Max_Hoffmann)\ **Post date:** [May 10, 2013, 2:12pm UTC](https://discuss.elastic.co/t/howto-access-character-offset-of-term-in-string-field/11914/1 "2013-05-10T14:12:27Z")
Dear All, I just started to play with elasticsearch and must say the usability is quite nice. However I really need a feature that seems not so trivial to access. Given my mapping curl -XGET '[http://localhost:9200/articles/](http://localhost:9200/articles/)' curl -XPUT '[http://localhost:9200/articles/article/\_mapping](http://localhost:9200/articles/article/_mapping)' -d '{ "articles" : { "properties" : { "source\_type" : {"type": "string", "store": "yes"}, "source\_id" : {"type": "string", "store": "yes"}, "body" : {"type": "string", "store": "yes", "term\_vector": "with\_positions\_offsets"}, "numbers": {"type": "boolean"} } } } ' The details of this mapping are not important only that I set "term\_vector": "with\_positions\_offsets" on the body field. I would like to access the explicit offsets of search terms in a string field to calculate a score. Please also note questions > and [http://stackoverflow.com/questions/15072806/elasticsearch-get-offsets-of-highlighted-snippets](http://stackoverflow.com/questions/15072806/elasticsearch-get-offsets-of-highlighted-snippets) . I think marking the raw text with a special token as pre\_tag/post\_tag to determine the position on the client sounds rather backwards if all I need is the explicit offset. Ideally one could access it in a script\_field evaluation. Please tell me if I am missing something obvious. If it is something elasticsearch just hack in on a regular basis, could someone point me in the right direction? Best, Max. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Max\_Hoffmann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/max_hoffmann/32/2356_2.png) [@Max\_Hoffmann](https://discuss.elastic.co/u/Max_Hoffmann)\ **Post date:** [May 28, 2013, 10:09pm UTC](https://discuss.elastic.co/t/howto-access-character-offset-of-term-in-string-field/11914/2 "2013-05-28T22:09:15Z")
Dear All I made some progress here by writing a plugin. I found [1] and [2] very useful here. Would like to share once I cleaned this up a bit. Now my only problem left is that the offsets from newly indexed documents do not show up until I restart elasticsearch, (i.e. reopen all readers). Since I have only little experience with elasticsearch: Could someone please describe whether it is possible to get real-time or near-real-time termvector/offset information in the plugin scope in a similarly automatic way as it work for search/get? Best, Max [1] [http://jfarrell.github.io/](http://jfarrell.github.io/) [2] [http://jprante.github.io/lessons/2012/03/27/Writing-a-simple-plugin-for-Elasticsearch.html](http://jprante.github.io/lessons/2012/03/27/Writing-a-simple-plugin-for-Elasticsearch.html) Am Freitag, 10. Mai 2013 16:12:27 UTC+2 schrieb Max Hoffmann: > Dear All, > > I just started to play with elasticsearch and must say the usability is > quite nice. However I really need a feature that seems not so trivial to > access. Given my mapping > > curl -XGET '[http://localhost:9200/articles/](http://localhost:9200/articles/)' > curl -XPUT '[http://localhost:9200/articles/article/\_mapping](http://localhost:9200/articles/article/_mapping)' -d '{ > "articles" : { > "properties" : { > "source\_type" : {"type": "string", "store": "yes"}, > "source\_id" : {"type": "string", "store": "yes"}, > "body" : {"type": "string", "store": "yes", "term\_vector": > "with\_positions\_offsets"}, > "numbers": {"type": "boolean"} > } > } > } > ' > > The details of this mapping are not important only that I set > "term\_vector": "with\_positions\_offsets" on the body field. I would like to > access the explicit offsets of search terms in a string field to calculate > a score. Please also note questions > [ElasticSearch get offsets of highlighted snippets - Stack Overflow](http://stackoverflow.com/questions/15072806/elasticsearch-get-offsets-of-highlighted-snippets) > and > [ElasticSearch get offsets of highlighted snippets - Stack Overflow](http://stackoverflow.com/questions/15072806/elasticsearch-get-offsets-of-highlighted-snippets) . > I think marking the raw text with a special token as pre\_tag/post\_tag to > determine the position on the client sounds rather backwards if all I need > is the explicit offset. Ideally one could access it in a script\_field > evaluation. > > Please tell me if I am missing something obvious. If it is something > elasticsearch just hack in on a regular basis, could someone point me in > the right direction? > > Best, > Max. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [May 28, 2013, 10:28pm UTC](https://discuss.elastic.co/t/howto-access-character-offset-of-term-in-string-field/11914/3 "2013-05-28T22:28:45Z")
My blog post is very old ☹ so no longer up to date. I assume you are on ES 0.90 (Lucene 4), so have you tried to access the reader with getLiveDocs()? [http://lucene.apache.org/core/4\_0\_0-BETA/MIGRATE.html](http://lucene.apache.org/core/4_0_0-BETA/MIGRATE.html) --\> DocsEnum docsEnum = reader.termDocsEnum(reader.getLiveDocs(), field, text, needsFreqs); Likewise for DocsAndPositionsEnum. Jörg Am 29.05.13 00:09, schrieb Max Hoffmann: > I made some progress here by writing a plugin. I found [1] and [2] > very useful here. Would like to share once I cleaned this up a bit. > Now my only problem left is that the offsets from newly indexed > documents do not show up until I restart elasticsearch, (i.e. reopen > all readers). Since I have only little experience with elasticsearch: > Could someone please describe whether it is possible to get real-time > or near-real-time termvector/offset information in the plugin scope in > a similarly automatic way as it work for search/get? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Max\_Hoffmann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/max_hoffmann/32/2356_2.png) [@Max\_Hoffmann](https://discuss.elastic.co/u/Max_Hoffmann)\ **Post date:** [May 29, 2013, 12:23pm UTC](https://discuss.elastic.co/t/howto-access-character-offset-of-term-in-string-field/11914/4 "2013-05-29T12:23:47Z")
Am Mittwoch, 29. Mai 2013 00:28:45 UTC+2 schrieb Jörg Prante: > My blog post is very old ☹ so no longer up to date. I assume you are Well, worked for me so far :-). Thanks. on ES 0.90 (Lucene 4), so have you tried to access the reader with > getLiveDocs()? > > [Apache Lucene Migration Guide](http://lucene.apache.org/core/4_0_0-BETA/MIGRATE.html) > > --\> DocsEnum docsEnum = reader.termDocsEnum(reader.getLiveDocs(), field, > text, needsFreqs); So, as far as I understood the documentation, reader.getLiveDocs() can only help to filter out deleted documents from a reader but not add newly indexed documents to an existing reader. Also it seems that elasticsearch 0.90 uses lucene 4.2 where the interface for termDocsEnum has changed[1]. Regardless, I am mostly wondering right now, whether it is possible to reopen a reader (if index changed) from the environment where the Transport\*Action.java is run. Best, Max. [1] [Lucene 4.2.0 API](http://lucene.apache.org/core/4_2_0/core/index.html) -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:34am UTC](https://discuss.elastic.co/t/howto-access-character-offset-of-term-in-string-field/11914/5 "2017-07-06T02:34:07Z")