# Text classification with ES

**URL:** <https://discuss.elastic.co/t/text-classification-with-es/130065>\
**Category:** Elasticsearch\
**Created:** [April 30, 2018, 9:07pm UTC](https://discuss.elastic.co/t/text-classification-with-es/130065 "2018-04-30T21:07:51Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![sanketshinde](https://avatars.discourse-cdn.com/v4/letter/s/b5ac83/32.png) [@sanketshinde](https://discuss.elastic.co/u/sanketshinde)\
**Post date:** [April 30, 2018, 9:07pm UTC](https://discuss.elastic.co/t/text-classification-with-es/130065/1 "2018-04-30T21:07:51Z")

</div>

Hello,

I followed the [blog](https://www.elastic.co/blog/text-classification-made-easy-with-elasticsearch) and ran the classification on the 20 News Group dataset, however I had a surprising experience. When tested the classification on the indexed documents as the test set, the accuracy was surprisingly low. I would like to know why is this the case. Here is some addition information.

Classification report:  
Class | Precision | Recall | F1-Score | Support

 ![11](https://us1.discourse-cdn.com/elastic/original/3X/1/8/18a6c82d1fc084e558de08e9a20856b6d774788f.png)  
mapping file:  
{  
"properties":{  
"content":{  
"type":"text",  
"analyzer":"english",  
"term\_vector": "yes"  
},  
"topic":{  
"type":"text",  
"analyzer":"english",  
"fields":{  
"raw":{  
"type":"keyword"  
}  
}  
}  
}  
}

MLT:

data = {  
"query":  
{  
"more\_like\_this": {  
"fields": [  
"content",  
"topic"  
],  
"like": "User text to classify",  
"min\_term\_freq": 1,  
"max\_query\_terms": 30  
}  
}  
}

---

<div class="post-metadata">

**Author:** ![sanketshinde](https://avatars.discourse-cdn.com/v4/letter/s/b5ac83/32.png) [@sanketshinde](https://discuss.elastic.co/u/sanketshinde)\
**Post date:** [May 2, 2018, 11:28am UTC](https://discuss.elastic.co/t/text-classification-with-es/130065/2 "2018-05-02T11:28:54Z")

</div>

In some more experimental runs that I made, I used different values for the min\_doc\_freq hyper-parameter for the MLT query and noticed that the accuracy improved. On a much smaller dataset, that I cannot make public, the accuracy on the training dataset improved from 60% to 92% when min\_doc\_freq was moved from default value 5 down to 1. The test set accuracy was 72% I think this is already commendable for the fact that it is for free and is very fast to set up. Great work guys!

Still, any insights, thoughts ?

---

<div class="post-metadata">

**Author:** ![softwaredoug](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/softwaredoug/32/22681_2.png) [@softwaredoug](https://discuss.elastic.co/u/softwaredoug)\
**Post date:** [May 2, 2018, 11:58am UTC](https://discuss.elastic.co/t/text-classification-with-es/130065/3 "2018-05-02T11:58:30Z")

</div>

I’m not sure about your corpus, but there’s a tricky balance eliminating terms/phrases where doc freq overstates how common they are. In other words, given the zipfian distribution of terms in a corpus, many, many (most?) terms occur just once. it’s difficult to know if a single term occurrence is “natural” or if it’s spurious and actually naturally more rare in the language.

So having a doc freq floor is often needed to ensure that indeed we can reliably say this is a “natural” rare term with confidence. But it’s a tricky balance. Most terms are rare, so get too aggressive and you’ll eliminate much of the vocabulary from your corpus.

It turns into a very corpus-specific exercise.

Ted Dunning has a classic paper “On Surprise and Coincidence” I highly recommend to understand term statistics. He has a method for gaining confidence in a term concurrence I find appealing.

This blog article of mine might also be useful

[https://opensourceconnections.com/blog/2016/03/29/semantic-search-with-latent-semantic-analysis/](https://opensourceconnections.com/blog/2016/03/29/semantic-search-with-latent-semantic-analysis/)

Doug

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 30, 2018, 11:58am UTC](https://discuss.elastic.co/t/text-classification-with-es/130065/4 "2018-05-30T11:58:34Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.