fieldNorm & queryNorm in explain api

Elastic Stack Elasticsearch
1

Hi,

I want to know what are fieldNorm and queryNorm in the output displayed by
the explain api..how are they calculated?

Thanks
Usha

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a009d8bd-6646-492e-8dc0-1af682a78286%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

The norms are calculated by Lucene's TFIDF algorithm:
http://lucene.apache.org/core/4_6_0/core/org/apache/lucene/search/similarities/TFIDFSimilarity.html

The field norm is encoded into a single byte, so there is a loss of
precision.

--
Ivan

On Tue, Feb 11, 2014 at 9:32 PM, usha2626@gmail.com wrote:

Hi,

I want to know what are fieldNorm and queryNorm in the output displayed by
the explain api..how are they calculated?

Thanks
Usha

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/a009d8bd-6646-492e-8dc0-1af682a78286%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQAzhPuEZ9H%2BN0pnC-xmWvscxitDie4%3DSVtjus_B%2Bp8GDA%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

Ivan
formula is queryNorm(q) = queryNorm(sumOfSquaredWeights)http://lucene.apache.org/core/4_6_0/core/org/apache/lucene/search/similarities/DefaultSimilarity.html#queryNorm(float) =
1 –––––––––––––– sumOfSquaredWeights½

The sum of squared weights (of the query terms) is computed by the query
Weighthttp://lucene.apache.org/core/4_6_0/core/org/apache/lucene/search/Weight.htmlobject. For example, a
BooleanQueryhttp://lucene.apache.org/core/4_6_0/core/org/apache/lucene/search/BooleanQuery.htmlcomputes this value as:

sumOfSquaredWeightshttp://lucene.apache.org/core/4_6_0/core/org/apache/lucene/search/Weight.html#getValueForNormalization() =
q.getBoost()http://lucene.apache.org/core/4_6_0/core/org/apache/lucene/search/Query.html#getBoost()
2 · ∑ ( idf(t)http://lucene.apache.org/core/4_6_0/core/org/apache/lucene/search/similarities/TFIDFSimilarity.html#formula_idf ·
t.getBoost()http://lucene.apache.org/core/4_6_0/core/org/apache/lucene/search/similarities/TFIDFSimilarity.html#formula_termBoost )
2
t in q

for my following search result how it is calculating:

{

  • took: 16
  • timed_out: false
  • _shards: {
    • total: 5
    • successful: 5
    • failed: 0
      }
  • hits: {
    • total: 2
    • max_score: 0.2712221
    • hits: [
      • {
        • _shard: 3
        • _node: k8BXmkARRsaaYlUJTRpIqQ
        • _index: phone
        • _type: iphone
        • _id: 2
        • _score: 0.2712221
        • _source: {
          • title: iphone update
          • description: this section shows how to update yopur iphone
            to work smoothly in your iphone
            }
        • _explanation: {
          • value: 0.27122214
          • description: max of:
          • details: [
            • {
              • value: 0.27122214
              • description: sum of:
              • details: [
                • {
                  • value: 0.13561107
                  • description: weight(title:iphone in 0)
                    [PerFieldSimilarity], result of:
                  • details: [
                    • {
                      • value: 0.13561107
                      • description: score(doc=0,freq=1.0 =
                        termFreq=1.0 ), product of:
                      • details: [
                        • {
                          • value: 0.70710677
                          • description: queryWeight, product
                            of:
                          • details: [
                            • {
                              • value: 0.30685282
                              • description: idf(docFreq=1,
                                maxDocs=1)
                                }
                            • {
                              • value: 2.3043842
                              • description: queryNorm
                                }
                                ]
                                }
                        • {
                          • value: 0.19178301
                          • description: fieldWeight in 0,
                            product of:
                          • details: [
                            • {
                              • value: 1
                              • description: tf(freq=1.0),
                                with freq of:
                              • details: [
                                • {
                                  • value: 1
                                  • description:
                                    termFreq=1.0
                                    }
                                    ]
                                    }
                            • {
                              • value: 0.30685282
                              • description: idf(docFreq=1,
                                maxDocs=1)
                                }
                            • {
                              • value: 0.625
                              • description: fieldNorm(doc=0)
                                }
                                ]
                                }
                                ]
                                }
                                ]
                                }
                • {
                  • value: 0.13561107
                  • description: weight(title:update in 0)
                    [PerFieldSimilarity], result of:
                  • details: [
                    • {
                      • value: 0.13561107
                      • description: score(doc=0,freq=1.0 =
                        termFreq=1.0 ), product of:
                      • details: [
                        • {
                          • value: 0.70710677
                          • description: queryWeight, product
                            of:
                          • details: [
                            • {
                              • value: 0.30685282
                              • description: idf(docFreq=1,
                                maxDocs=1)
                                }
                            • {
                              • value: 2.3043842
                              • description: queryNorm
                                }
                                ]
                                }
                        • {
                          • value: 0.19178301
                          • description: fieldWeight in 0,
                            product of:
                          • details: [
                            • {
                              • value: 1
                              • description: tf(freq=1.0),
                                with freq of:
                              • details: [
                                • {
                                  • value: 1
                                  • description:
                                    termFreq=1.0
                                    }
                                    ]
                                    }
                            • {
                              • value: 0.30685282
                              • description: idf(docFreq=1,
                                maxDocs=1)
                                }
                            • {
                              • value: 0.625
                              • description: fieldNorm(doc=0)
                                }
                                ]
                                }
                                ]
                                }
                                ]
                                }
                                ]
                                }
            • {
              • value: 0.16369703
              • description: sum of:
              • details: [
                • {
                  • value: 0.0958915
                  • description: weight(description:iphone in 0)
                    [PerFieldSimilarity], result of:
                  • details: [
                    • {
                      • value: 0.0958915
                      • description: score(doc=0,freq=2.0 =
                        termFreq=2.0 ), product of:
                      • details: [
                        • {
                          • value: 0.70710677
                          • description: queryWeight, product
                            of:
                          • details: [
                            • {
                              • value: 0.30685282
                              • description: idf(docFreq=1,
                                maxDocs=1)
                                }
                            • {
                              • value: 2.3043842
                              • description: queryNorm
                                }
                                ]
                                }
                        • {
                          • value: 0.13561106
                          • description: fieldWeight in 0,
                            product of:
                          • details: [
                            • {
                              • value: 1.4142135
                              • description: tf(freq=2.0),
                                with freq of:
                              • details: [
                                • {
                                  • value: 2
                                  • description:
                                    termFreq=2.0
                                    }
                                    ]
                                    }
                            • {
                              • value: 0.30685282
                              • description: idf(docFreq=1,
                                maxDocs=1)
                                }
                            • {
                              • value: 0.3125
                              • description: fieldNorm(doc=0)
                                }
                                ]
                                }
                                ]
                                }
                                ]
                                }
                • {
                  • value: 0.067805536
                  • description: weight(description:update in 0)
                    [PerFieldSimilarity], result of:
                  • details: [
                    • {
                      • value: 0.067805536
                      • description: score(doc=0,freq=1.0 =
                        termFreq=1.0 ), product of:
                      • details: [
                        • {
                          • value: 0.70710677
                          • description: queryWeight, product
                            of:
                          • details: [
                            • {
                              • value: 0.30685282
                              • description: idf(docFreq=1,
                                maxDocs=1)
                                }
                            • {
                              • value: 2.3043842
                              • description: queryNorm
                                }
                                ]
                                }
                        • {
                          • value: 0.095891505
                          • description: fieldWeight in 0,
                            product of:
                          • details: [
                            • {
                              • value: 1
                              • description: tf(freq=1.0),
                                with freq of:
                              • details: [
                                • {
                                  • value: 1
                                  • description:
                                    termFreq=1.0
                                    }
                                    ]
                                    }
                            • {
                              • value: 0.30685282
                              • description: idf(docFreq=1,
                                maxDocs=1)
                                }
                            • {
                              • value: 0.3125
                              • description: fieldNorm(doc=0)
                                }
                                ]
                                }
                                ]
                                }
                                ]
                                }
                                ]
                                }
                                ]
                                }
                                }
      • {
        • _shard: 2
        • _node: k8BXmkARRsaaYlUJTRpIqQ
        • _index: phone
        • _type: iphone
        • _id: 1
        • _score: 0.04500804
        • _source: {
          • title: iphone
          • description: welcome to iphone world
            }
        • _explanation: {
          • value: 0.04500804
          • description: max of:
          • details: [
            • {
              • value: 0.04500804
              • description: product of:
              • details: [
                • {
                  • value: 0.09001608
                  • description: sum of:
                  • details: [
                    • {
                      • value: 0.09001608
                      • description: weight(title:iphone in 0)
                        [PerFieldSimilarity], result of:
                      • details: [
                        • {
                          • value: 0.09001608
                          • description: score(doc=0,freq=1.0
                            = termFreq=1.0 ), product of:
                          • details: [
                            • {
                              • value: 0.29335263
                              • description: queryWeight,
                                product of:
                              • details: [
                                • {
                                  • value: 0.30685282
                                  • description: idf(docFreq=1,
                                    maxDocs=1)
                                    }
                                • {
                                  • value: 0.9560043
                                  • description: queryNorm
                                    }
                                    ]
                                    }
                            • {
                              • value: 0.30685282
                              • description: fieldWeight in
                                0, product of:
                              • details: [
                                • {
                                  • value: 1
                                  • description: tf(freq=1.0),
                                    with freq of:
                                  • details: [
                                    • {
                                      • value: 1
                                      • description:
                                        termFreq=1.0
                                        }
                                        ]
                                        }
                                • {
                                  • value: 0.30685282
                                  • description: idf(docFreq=1,
                                    maxDocs=1)
                                    }
                                • {
                                  • value: 1
                                  • description:
                                    fieldNorm(doc=0)
                                    }
                                    ]
                                    }
                                    ]
                                    }
                                    ]
                                    }
                                    ]
                                    }
                • {
                  • value: 0.5
                  • description: coord(1/2)
                    }
                    ]
                    }
            • {
              • value: 0.02250402
              • description: product of:
              • details: [
                • {
                  • value: 0.04500804
                  • description: sum of:
                  • details: [
                    • {
                      • value: 0.04500804
                      • description: weight(description:iphone
                        in 0) [PerFieldSimilarity], result of:
                      • details: [
                        • {
                          • value: 0.04500804
                          • description: score(doc=0,freq=1.0
                            = termFreq=1.0 ), product of:
                          • details: [
                            • {
                              • value: 0.29335263
                              • description: queryWeight,
                                product of:
                              • details: [
                                • {
                                  • value: 0.30685282
                                  • description: idf(docFreq=1,
                                    maxDocs=1)
                                    }
                                • {
                                  • value: 0.9560043
                                  • description: queryNorm
                                    }
                                    ]
                                    }
                            • {
                              • value: 0.15342641
                              • description: fieldWeight in
                                0, product of:
                              • details: [
                                • {
                                  • value: 1
                                  • description: tf(freq=1.0),
                                    with freq of:
                                  • details: [
                                    • {
                                      • value: 1
                                      • description:
                                        termFreq=1.0
                                        }
                                        ]
                                        }
                                • {
                                  • value: 0.30685282
                                  • description: idf(docFreq=1,
                                    maxDocs=1)
                                    }
                                • {
                                  • value: 0.5
                                  • description:
                                    fieldNorm(doc=0)
                                    }
                                    ]
                                    }
                                    ]
                                    }
                                    ]
                                    }
                                    ]
                                    }
                • {
                  • value: 0.5
                  • description: coord(1/2)
                    }
                    ]
                    }
                    ]
                    }
                    }
                    ]
                    }

}

On Wednesday, 12 February 2014 12:00:03 UTC+5:30, Ivan Brusic wrote:

The norms are calculated by Lucene's TFIDF algorithm:
TFIDFSimilarity (Lucene 4.6.0 API)

The field norm is encoded into a single byte, so there is a loss of
precision.

--
Ivan

On Tue, Feb 11, 2014 at 9:32 PM, <usha...@gmail.com <javascript:>> wrote:

Hi,

I want to know what are fieldNorm and queryNorm in the output displayed
by the explain api..how are they calculated?

Thanks
Usha

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/a009d8bd-6646-492e-8dc0-1af682a78286%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/4d86972e-6cca-488f-97d2-e9a1edef752f%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.