# Text analysis

**URL:** <https://discuss.elastic.co/t/text-analysis/171696>\
**Category:** Elasticsearch\
**Created:** [March 11, 2019, 8:16am UTC](https://discuss.elastic.co/t/text-analysis/171696 "2019-03-11T08:16:21Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Priyasmini\_Sahoo](https://avatars.discourse-cdn.com/v4/letter/p/b782af/32.png) [@Priyasmini\_Sahoo](https://discuss.elastic.co/u/Priyasmini_Sahoo)\
**Post date:** [March 11, 2019, 8:16am UTC](https://discuss.elastic.co/t/text-analysis/171696/1 "2019-03-11T08:16:21Z")

</div>

I have an index in Elasticsearch. It has fields with string values. Can I tokenize them? or Can I do text analysis in that index?

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [March 11, 2019, 8:18am UTC](https://discuss.elastic.co/t/text-analysis/171696/2 "2019-03-11T08:18:57Z")

</div>

> [@Priyasmini\_Sahoo](#):
>
> Can I tokenize them?

It's done by default at index and search time.  
What do you mean? What is your need exactly?

---

<div class="post-metadata">

**Author:** ![Priyasmini\_Sahoo](https://avatars.discourse-cdn.com/v4/letter/p/b782af/32.png) [@Priyasmini\_Sahoo](https://discuss.elastic.co/u/Priyasmini_Sahoo)\
**Post date:** [March 11, 2019, 8:25am UTC](https://discuss.elastic.co/t/text-analysis/171696/3 "2019-03-11T08:25:49Z")

</div>

I wanted to do sentiment analysis of Twitter data. I tried to do standard tokenization by typing in dev tools like this:-  
PUT /priya\_twitter\_pok  
{  
"settings": {  
"number\_of\_shards": 1,  
"analysis": {  
"filter": {  
"autocomplete\_filter": {  
"type": "edge\_ngram",  
"min\_gram": 1,  
"max\_gram": 20  
}  
},  
"analyzer": {  
"autocomplete": {  
"type": "custom",  
"tokenizer": "standard",  
"filter": [  
"lowercase",  
"autocomplete\_filter"  
]  
}  
}  
}  
}  
}  
but error is coming like this:-  
{  
"error": {  
"root\_cause": [  
{  
"type": "resource\_already\_exists\_exception",  
"reason": "index [priya\_twitter\_pok/iyshYCcuSA6CQ\_OsKEShHg] already exists",  
"index\_uuid": "iyshYCcuSA6CQ\_OsKEShHg",  
"index": "priya\_twitter\_pok"  
}  
],  
"type": "resource\_already\_exists\_exception",  
"reason": "index [priya\_twitter\_pok/iyshYCcuSA6CQ\_OsKEShHg] already exists",  
"index\_uuid": "iyshYCcuSA6CQ\_OsKEShHg",  
"index": "priya\_twitter\_pok"  
},  
"status": 400  
}

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [March 11, 2019, 8:33am UTC](https://discuss.elastic.co/t/text-analysis/171696/4 "2019-03-11T08:33:06Z")

</div>

Please format your code, logs or configuration files using `</>` icon as explained in [this guide](https://discuss.elastic.co/t/about-the-elasticsearch-category/21) and not the citation button. It will make your post more readable.

Or use markdown style like:

````
```
CODE
```

````

This is the icon to use if you are not using markdown format:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/7/e/7e6e239431ec2d71cbf1beef741f2e93e7cc762c.jpg)

There's a live preview panel for exactly this reasons.

Lots of people read these forums, and many of them will simply skip over a post that is difficult to read, because it's just too large an investment of their time to try and follow a wall of badly formatted text.  
If your goal is to get an answer to your questions, it's in your interest to make it as easy to read and understand as possible.  
Please update your post.

Add:

```auto
DELETE /priya_twitter_pok

```

At the beginning at your script.

---

<div class="post-metadata">

**Author:** ![Priyasmini\_Sahoo](https://avatars.discourse-cdn.com/v4/letter/p/b782af/32.png) [@Priyasmini\_Sahoo](https://discuss.elastic.co/u/Priyasmini_Sahoo)\
**Post date:** [March 11, 2019, 8:56am UTC](https://discuss.elastic.co/t/text-analysis/171696/5 "2019-03-11T08:56:40Z")

</div>

Sorry for the bad formatting. I have written the query in Dev Tools as follows:-

```
        PUT /priya_twitter_pok
        {
          "settings": {
            "number_of_shards": 1,
            "analysis": {
              "filter": {
                "autocomplete_filter": {
                  "type": "edge_ngram",
                  "min_gram": 1,
                  "max_gram": 20
                }
              },
              "analyzer": {
                "autocomplete": {
                  "type": "custom",
                  "tokenizer": "standard",
                  "filter": [
                    "lowercase",
                    "autocomplete_filter"
                  ]
                }
              }
            }
          }
        }
    `

```

After executing the above query error is coming as follows:-

```
{
  "error": {
    "root_cause": [
      {
        "type": "resource_already_exists_exception",
        "reason": "index [priya_twitter_pok/iyshYCcuSA6CQ_OsKEShHg] already exists",
        "index_uuid": "iyshYCcuSA6CQ_OsKEShHg",
        "index": "priya_twitter_pok"
      }
    ],
    "type": "resource_already_exists_exception",
    "reason": "index [priya_twitter_pok/iyshYCcuSA6CQ_OsKEShHg] already exists",
    "index_uuid": "iyshYCcuSA6CQ_OsKEShHg",
    "index": "priya_twitter_pok"
  },
  "status": 400
}

```

And can you tell me why I would have to _ **DELETE** _ my index before beginning my script?

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [March 11, 2019, 9:38am UTC](https://discuss.elastic.co/t/text-analysis/171696/6 "2019-03-11T09:38:00Z")

</div>

Because your index `priya_twitter_pok` already exists.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 8, 2019, 9:38am UTC](https://discuss.elastic.co/t/text-analysis/171696/7 "2019-04-08T09:38:01Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.