# Different Length Strings with Same FieldNorm

**URL:** <https://discuss.elastic.co/t/different-length-strings-with-same-fieldnorm/6224>\
**Category:** Elasticsearch\
**Created:** [December 22, 2011, 6:53pm UTC](https://discuss.elastic.co/t/different-length-strings-with-same-fieldnorm/6224 "2011-12-22T18:53:52Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![James\_Bathgate](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/james_bathgate/32/1758_2.png) [@James\_Bathgate](https://discuss.elastic.co/u/James_Bathgate)\
**Post date:** [December 22, 2011, 6:53pm UTC](https://discuss.elastic.co/t/different-length-strings-with-same-fieldnorm/6224/1 "2011-12-22T18:53:52Z")

</div>

I'm having an issue when using an nGram filter with the fieldNorm of two  
fields with different length strings coming back with the same fieldNorm.  
I've written a base simple test case for review which you can see below. I  
just can't seem to figure out why "ktm monster graphics" is getting the  
same score as "monster graphics". Can anyone shed any light on this?

echo "Deleting old index... ";

curl -XDELETE 'localhost:9200/jtest/';

echo -e "\r";

echo "Creating new index... ";

curl -XPUT '[http://localhost:9200/jtest/](http://localhost:9200/jtest/)' -d '{  
"settings" : {  
"index" : {  
"number\_of\_shards" : "1",  
"number\_of\_replicas" : 0  
},  
"analysis" : {  
"analyzer" : {  
"iVal" : {  
"type" : "custom",  
"tokenizer" : "whitespace",  
"filter" : ["lowercase", "partial"]  
},  
"sVal" : {  
"type" : "custom",  
"tokenizer" : "whitespace",  
"filter" : ["lowercase"]  
}  
},  
"filter" : {  
"partial" : {  
"type" : "edgeNGram",  
"min\_gram" : "2",  
"max\_gram" : "20"  
}  
}  
}  
},  
"mappings" : {  
"test\_type" : {  
"properties" : {  
"test\_value" : {  
"type" : "string",  
"store" : true,  
"index" : "analyzed",  
"index\_analyzer" : "iVal",  
"search\_analyzer" : "sVal"  
}  
}  
}  
}  
}';

echo -e "\r";

echo "Inserting Data...";

curl -XPUT '[http://localhost:9200/jtest/test\_type/2](http://localhost:9200/jtest/test_type/2)' -d '{  
"test\_value" : "ktm monster graphics"  
}';

echo -e "\r";

curl -XPUT '[http://localhost:9200/jtest/test\_type/1](http://localhost:9200/jtest/test_type/1)' -d '{  
"test\_value" : "monster graphics"  
}';

echo -e "\r";

sleep 5

echo "Performing Search Query...";

curl -XGET '[http://localhost:9200/jtest/test\_type/\_search?pretty=1](http://localhost:9200/jtest/test_type/_search?pretty=1)' -d '{  
"query" : {  
"text" : {  
"test\_value" : "monster graphics"  
}  
}  
}';

echo -e "\r";

---

<div class="post-metadata">

**Author:** ![James\_Bathgate](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/james_bathgate/32/1758_2.png) [@James\_Bathgate](https://discuss.elastic.co/u/James_Bathgate)\
**Post date:** [December 23, 2011, 3:21pm UTC](https://discuss.elastic.co/t/different-length-strings-with-same-fieldnorm/6224/2 "2011-12-23T15:21:36Z")

</div>

Just in case anyone wants the results of that last query with an explain.

{  
"took" : 2,  
"timed\_out" : false,  
"\_shards" : {  
"total" : 1,  
"successful" : 1,  
"failed" : 0  
},  
"hits" : {  
"total" : 2,  
"max\_score" : 0.10509991,  
"hits" : [ {  
"\_shard" : 0,  
"\_node" : "nPbYBu5UQquwUEVVht03CA",  
"\_index" : "jtest",  
"\_type" : "test\_type",  
"\_id" : "2",  
"\_score" : 0.10509991, "\_source" : {  
"test\_value" : "ktm monster graphics"  
},  
"\_explanation" : {  
"value" : 0.10509991,  
"description" : "sum of:",  
"details" : [ {  
"value" : 0.052549955,  
"description" : "weight(test\_value:monster in 0), product of:",  
"details" : [ {  
"value" : 0.70710677,  
"description" : "queryWeight(test\_value:monster), product of:",  
"details" : [ {  
"value" : 0.5945349,  
"description" : "idf(docFreq=2, maxDocs=2)"  
}, {  
"value" : 1.1893445,  
"description" : "queryNorm"  
} ]  
}, {  
"value" : 0.07431686,  
"description" : "fieldWeight(test\_value:monster in 0), product  
of:",  
"details" : [ {  
"value" : 1.0,  
"description" : "tf(termFreq(test\_value:monster)=1)"  
}, {  
"value" : 0.5945349,  
"description" : "idf(docFreq=2, maxDocs=2)"  
}, {  
"value" : 0.125,  
"description" : "fieldNorm(field=test\_value, doc=0)"  
} ]  
} ]  
}, {  
"value" : 0.052549955,  
"description" : "weight(test\_value:graphics in 0), product of:",  
"details" : [ {  
"value" : 0.70710677,  
"description" : "queryWeight(test\_value:graphics), product of:",  
"details" : [ {  
"value" : 0.5945349,  
"description" : "idf(docFreq=2, maxDocs=2)"  
}, {  
"value" : 1.1893445,  
"description" : "queryNorm"  
} ]  
}, {  
"value" : 0.07431686,  
"description" : "fieldWeight(test\_value:graphics in 0), product  
of:",  
"details" : [ {  
"value" : 1.0,  
"description" : "tf(termFreq(test\_value:graphics)=1)"  
}, {  
"value" : 0.5945349,  
"description" : "idf(docFreq=2, maxDocs=2)"  
}, {  
"value" : 0.125,  
"description" : "fieldNorm(field=test\_value, doc=0)"  
} ]  
} ]  
} ]  
}  
}, {  
"\_shard" : 0,  
"\_node" : "nPbYBu5UQquwUEVVht03CA",  
"\_index" : "jtest",  
"\_type" : "test\_type",  
"\_id" : "1",  
"\_score" : 0.10509991, "\_source" : {  
"test\_value" : "monster graphics"  
},  
"\_explanation" : {  
"value" : 0.10509991,  
"description" : "sum of:",  
"details" : [ {  
"value" : 0.052549955,  
"description" : "weight(test\_value:monster in 1), product of:",  
"details" : [ {  
"value" : 0.70710677,  
"description" : "queryWeight(test\_value:monster), product of:",  
"details" : [ {  
"value" : 0.5945349,  
"description" : "idf(docFreq=2, maxDocs=2)"  
}, {  
"value" : 1.1893445,  
"description" : "queryNorm"  
} ]  
}, {  
"value" : 0.07431686,  
"description" : "fieldWeight(test\_value:monster in 1), product  
of:",  
"details" : [ {  
"value" : 1.0,  
"description" : "tf(termFreq(test\_value:monster)=1)"  
}, {  
"value" : 0.5945349,  
"description" : "idf(docFreq=2, maxDocs=2)"  
}, {  
"value" : 0.125,  
"description" : "fieldNorm(field=test\_value, doc=1)"  
} ]  
} ]  
}, {  
"value" : 0.052549955,  
"description" : "weight(test\_value:graphics in 1), product of:",  
"details" : [ {  
"value" : 0.70710677,  
"description" : "queryWeight(test\_value:graphics), product of:",  
"details" : [ {  
"value" : 0.5945349,  
"description" : "idf(docFreq=2, maxDocs=2)"  
}, {  
"value" : 1.1893445,  
"description" : "queryNorm"  
} ]  
}, {  
"value" : 0.07431686,  
"description" : "fieldWeight(test\_value:graphics in 1), product  
of:",  
"details" : [ {  
"value" : 1.0,  
"description" : "tf(termFreq(test\_value:graphics)=1)"  
}, {  
"value" : 0.5945349,  
"description" : "idf(docFreq=2, maxDocs=2)"  
}, {  
"value" : 0.125,  
"description" : "fieldNorm(field=test\_value, doc=1)"  
} ]  
} ]  
} ]  
}  
} ]  
}  
}

---

<div class="post-metadata">

**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\
**Post date:** [December 23, 2011, 3:38pm UTC](https://discuss.elastic.co/t/different-length-strings-with-same-fieldnorm/6224/3 "2011-12-23T15:38:02Z")

</div>

On Thu, 2011-12-22 at 10:53 -0800, James Bathgate wrote:

> I'm having an issue when using an nGram filter with the fieldNorm of  
> two fields with different length strings coming back with the same  
> fieldNorm. I've written a base simple test case for review which you  
> can see below. I just can't seem to figure out why "ktm monster  
> graphics" is getting the same score as "monster graphics". Can anyone  
> shed any light on this?

Field norms are stored in 8 bits, so small differences in field length,  
when rounded down to 8 bits, are considered to be equal.

If you make the longer string just a bit longer (eg 'ktmxy monster  
graphics') then you see a difference:

"hits" : {  
"total" : 2,  
"max\_score" : 0.21019982,  
"hits" : [ {  
"\_index" : "jtest",  
"\_type" : "test\_type",  
"\_id" : "1",  
"\_score" : 0.21019982, "\_source" : {  
"test\_value" : "monster graphics"  
}  
}, {  
"\_index" : "jtest",  
"\_type" : "test\_type",  
"\_id" : "2",  
"\_score" : 0.18392484, "\_source" : {  
"test\_value" : "ktmxy monster graphics"  
}  
} ]  
}

clint

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 3:44am UTC](https://discuss.elastic.co/t/different-length-strings-with-same-fieldnorm/6224/4 "2017-07-06T03:44:25Z")

</div>