# Highlight the result of tokenization when viewing full text **URL:** \ **Category:** Elasticsearch\ **Created:** [February 13, 2019, 9:02pm UTC](https://discuss.elastic.co/t/highlight-the-result-of-tokenization-when-viewing-full-text/168292 "2019-02-13T21:02:30Z")\ **Posts on this page:** 2\ **Page:** 1
**Author:** ![1fmusic](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/1fmusic/32/37903_2.png) [@1fmusic](https://discuss.elastic.co/u/1fmusic)\ **Post date:** [February 13, 2019, 9:02pm UTC](https://discuss.elastic.co/t/highlight-the-result-of-tokenization-when-viewing-full-text/168292/1 "2019-02-13T21:02:30Z")
I have documents with duplicate sentences in the 'notes' field. I was able to tokenize this field and get only the original sentences and their offsets. When the user views this "notes" field, I would like to highlight these original sentences. It seems like I should be able to since the offsets are stored, but I just can't figure out how to implement. Any input on this matter is greatly appreciated. thank you. ``` // PUT mimic_dat { "settings": { "number_of_shards": 1, "number_of_replicas": 0, "analysis": { "tokenizer": { "mimic_tokenizer": { "type": "pattern", "pattern": """(\.\s|\n+)""", "group": -1 } }, "filter": { "unique_mimic": { "type": "unique", "only_on_same_position": false } }, "analyzer": { "mimic_hash_analyzer": { "type": "custom", "tokenizer": "mimic_tokenizer", "filter": [ "unique_mimic" ] } } } }, "mappings": { "mimic_type": { "properties": { "subject_id": { "type": "keyword" }, "notes": { "type": "text", "fielddata": true, "fields": { "my_hash": { "type": "text", "analyzer": "mimic_hash_analyzer", "fielddata": true, "term_vector": "with_positions_offsets", "store": true } } } } } } } // PUT mimic_dat/mimic_type/4 { "notes": """ Past History: Chronic xx which lead to; Ca. Review of systems: Cardiac, SR. O2: sats on room air 100%. ID: No active issues, temp 99.3 PO. Review of systems: Cardiac, SR. ID: No active issues, temp 99.3 PO. """ } ``` Reply ### This topic will close a month after the last reply. Bookmark Share Flag Reply Watching You will receive notifications because you created this topic. ### Suggested Topics | Topic | Replies | Views | Activity | | --- | --- | --- | --- | | [Run remote commands from DevTools](https://discuss.elastic.co/t/run-remote-commands-from-devtools/168290) | | | | [Elasticsearch](https://discuss.elastic.co/c/elasticsearch)|[0](https://discuss.elastic.co/t/return-text-with-duplicate-tokens-highlighted/166831)|2|[20m](https://discuss.elastic.co/t/run-remote-commands-from-devtools/168290/1)| |[Modify core storage of Elastic search](https://discuss.elastic.co/t/modify-core-storage-of-elastic-search/168274) [Elasticsearch](https://discuss.elastic.co/c/elasticsearch)|[2](https://discuss.elastic.co/t/return-text-with-duplicate-tokens-highlighted/166831)|16|[20m](https://discuss.elastic.co/t/modify-core-storage-of-elastic-search/168274/3)| |[ELK architecture optimization](https://discuss.elastic.co/t/elk-architecture-optimization/168282) [Elasticsearch](https://discuss.elastic.co/c/elasticsearch)|[0](https://discuss.elastic.co/t/return-text-with-duplicate-tokens-highlighted/166831)|6|[1h](https://discuss.elastic.co/t/elk-architecture-optimization/168282/1)| |[Recognizing succeeded vs failed tasks w/ the Task Management API](https://discuss.elastic.co/t/recognizing-succeeded-vs-failed-tasks-w-the-task-management-api/168281) [Elasticsearch](https://discuss.elastic.co/c/elasticsearch)|[0](https://discuss.elastic.co/t/return-text-with-duplicate-tokens-highlighted/166831)|7|[1h](https://discuss.elastic.co/t/recognizing-succeeded-vs-failed-tasks-w-the-task-management-api/168281/1)| |[Shipping logs from Central machine(Jenkins machine) to Elasticsearch](https://discuss.elastic.co/t/shipping-logs-from-central-machine-jenkins-machine-to-elasticsearch/168280) [Elasticsearch](https://discuss.elastic.co/c/elasticsearch)|[0](https://discuss.elastic.co/t/return-text-with-duplicate-tokens-highlighted/166831)|6|[1h](https://discuss.elastic.co/t/shipping-logs-from-central-machine-jenkins-machine-to-elasticsearch/168280/1)| ### There are [195 new](https://discuss.elastic.co/new) topics remaining, or browse other topics in [Elasticsearch](https://discuss.elastic.co/c/elasticsearch) © 2018. All Rights Reserved - Elasticsearch - Elasticsearch is a trademark of Elasticsearch BV, registered in the U.S. and in other countries - [Trademarks](https://www.elastic.co/legal/trademarks) - [Terms](https://www.elastic.co/legal/terms-of-use) - [Privacy](https://www.elastic.co/legal/privacy-policy) - [Brand](https://www.elastic.co/brand) - [Code of Conduct](https://www.elastic.co/community/codeofconduct) Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the [Apache Software Foundation](http://www.apache.org/) in the United States and/or other countries. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [March 13, 2019, 9:02pm UTC](https://discuss.elastic.co/t/highlight-the-result-of-tokenization-when-viewing-full-text/168292/2 "2019-03-13T21:02:31Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.