Access inner_hits response in Apache Hadoop Spark

Elastic Stack Elasticsearch
1

I have an Elasticsearch query that is giving me a response with standard _source data and some additional response that is inner_hits matching the nested document of the index. The response is something like below

{
  "took" : 9,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 5,
      "relation" : "eq"
    },
    "max_score" : 3.638414,
    "hits" : [
      {
        "_index" : "family",
        "_id" : "39684609",
        "_score" : 3.638414,
        "_source" : {...},
        "inner_hits" : {
          "representative.invention_title" : {
            "hits" : {
              "total" : {
                "value" : 1,
                "relation" : "eq"
              },
              "max_score" : 0.089231126,
              "hits" : [
                {
                  "_index" : "family",
                  "_id" : "39684609",
                  "_nested" : {
                    "field" : "representative.invention_title",
                    "offset" : 0
                  },
                  "_score" : 0.089231126,
                  "_source" : {
                    "title" : "Method for making a hammer"
                  }
                }
              ]
            }
          }
        }
      }
        ]
  }
}

I want to access that hits inside inner_hits. I can access source inside the spark dataframe, but not the inner_hits. I checked out the documentation they support metadata but I couldn't able to find a way to access inner_hits from metadata

2

Hi @anand_tripathi. I suspect it's not supported based on this comment: support highlighting and other non _source headers · Issue #460 · elastic/elasticsearch-hadoop · GitHub, plus the fact that I don't see any relevant test cases in the source code. Would you mind posting enough to reproduce this (commands to insert data into elasticsearch, plus spark code to query it)?

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.