How to get tokens for a document?

[VidyVideni]

I need to get all tokens for a document when debug, so I know what exact tokens indexed in elasticsearch , this can help me understand why a query doesn't get any result when it supposed to be . any way to do this please?

[effendisusanto]

Hi Vidy,

Yesterday I asked the same question, you can read the answer on the elasticsearch guide,

https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-termvectors.html

Based on that guide, you can retrive the token on document by the following query :

GET /twitter/tweet/1/_termvectors
{
"fields" : ["text"],
"offsets" : true,
"payloads" : true,
"positions" : true,
"term_statistics" : true,
"field_statistics" : true
}

Hope it save your night