ElasticSearch InnerHits Java API causing NPE


(Brent Evans) #1

I'm currently using the InnerHits API (In Java) to return some child documents, and it appears that the documents being returned fail to have a shard set on the SearchHit (Not the Inner document SearchHits, but the document type being returned).

The first SearchHit returned on the SearchResponse has a shard set on it, with all further SearchHits having the shard set to null. This only occurs if the search that was constructed has InnerHits set.

I've gone through release notes and have been unable to find any reference to this change.

Going through the ElasticSearch source briefly makes me think this is a bug in the InternalSearchHit class in the readFrom function (line 556) as the ShardTargetType is set to NO_STREAM when the first set of InnerHits are processed, with this causing the subsequent non-InnerHits to not set the shard.

Can anyone confirm if this is indeed a bug, or if it is intended?

This is on version 1.6.0.

Thanks,

Brent


(Brent Evans) #2

I've added a basic test program below which reproduces this.

The Parent and Child documents are very basic, both having just a simple Title text field, with the child document having the parent set as the parent (routing etc. is all configured correctly).

Running the test returns the following:

Parent1 [ID]
[UqLUWssUTcqaQsYRhdjSVA][test-index][0] [Shard]
Parent2 [ID]
[UqLUWssUTcqaQsYRhdjSVA][test-index][0] [Shard]

Parent1 [ID]
[UqLUWssUTcqaQsYRhdjSVA][test-index][0] [Shard]
Parent2 [ID]
null [Shard]

As you can see, the second document returns a null shard value when an inner hit is being used in the search. I could understand if it was the inner hit that didn't have a shard set, however this is a document of the result type, which has a shard set if inner hits aren't used (the size of the result doesn't make a difference, the shard is only ever being set on the first document in the SearchHits).

package com.main.elasticsearch;

import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.Client;
import org.elasticsearch.client.transport.TransportClient;
import org.elasticsearch.common.settings.ImmutableSettings;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.common.transport.InetSocketTransportAddress;
import org.elasticsearch.index.query.BoolQueryBuilder;
import org.elasticsearch.index.query.FilterBuilders;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.index.query.support.QueryInnerHitBuilder;
import org.elasticsearch.search.SearchHit;

public class ElasticInnerHitsTest {

	public static void main(String[] args) {
		Settings settings = ImmutableSettings.settingsBuilder().put("cluster.name", "testcluster").build();
		
		Client client = new TransportClient(settings)
				.addTransportAddress(new InetSocketTransportAddress("localhost", 9300));
		
		
		BoolQueryBuilder query = QueryBuilders.boolQuery().must(QueryBuilders.filteredQuery(QueryBuilders.matchAllQuery(), 
				FilterBuilders.hasChildFilter("Child", QueryBuilders.matchAllQuery())));
		
		SearchResponse response = client.prepareSearch("test-index")
				.setTypes("Parent")
				.setQuery(query)
				.setFrom(0)
				.setSize(2)
				.execute()
				.actionGet();
		
		for (SearchHit hit : response.getHits()) {
			System.out.println(hit.getId());
			System.out.println(hit.getShard());
		}
		
		QueryInnerHitBuilder innerHit = new QueryInnerHitBuilder();
		
		BoolQueryBuilder innerQuery = QueryBuilders.boolQuery().must(QueryBuilders.filteredQuery(QueryBuilders.matchAllQuery(), 
				FilterBuilders.hasChildFilter("Child", QueryBuilders.matchAllQuery())
				.innerHit(innerHit)));
		
		response = client.prepareSearch("test-index")
				.setTypes("Parent")
				.setQuery(innerQuery)
				.setFrom(0)
				.setSize(2)
				.execute()
				.actionGet();
		
		for (SearchHit hit : response.getHits()) {
			System.out.println(hit.getId());
			System.out.println(hit.getShard());
		}
		
		client.close();
	}

}

(Martijn Van Groningen) #3

Thanks for bringing this up! The shard is only set on the first search hit and not on the subsequent hits. I'll fix this soon.


(Martijn Van Groningen) #4

I opened this PR: https://github.com/elastic/elasticsearch/pull/12261

This change in behaviour was intended and each top level search hit needs to have a shard target.


(Brent Evans) #5

Thanks a lot.


(system) #6