SearchHit.fields() returns null even though fields are there


(John Chang) #1

Just upgraded from 0.9 to 0.11.0 and I find that org.elasticsearch.search.SearchHit.fields() is returning null even when I know the requested fields are there. When I do the query from the url through curl (including the requested fields), they show up fine in the returned json.

How I query:
response = indexClient.search(Requests.searchRequest(getIndexName()).types(documentTypes).searchType(SearchType.QUERY_THEN_FETCH)
.source(SearchSourceBuilder.searchSource().query(QueryBuilders.customScoreQuery(QueryBuilders.queryString(query)).script("doc['" + sortField + "'].value")).fields(fields).from(0).size(maxHitsPerIndexQuery).explain(true))

How I get the results:
List<Map<String, List>> results = new ArrayList<Map<String, List>>();
for(SearchHit hit : response.hits().hits()) {

        Map<String, List<Object>> fieldMap = new HashMap<String, List<Object>>();

        for (Map.Entry<String, SearchHitField> field : hit.fields().entrySet()) {  //NPE here
            fieldMap.put(field.getKey(), field.getValue().getValues()); 
        }

        results.add(fieldMap);
    }

Note this is not a problem when I start local nodes in my unit tests; I only see this problem when I connect to a true stand-alone node started from the command line.

Thanks.


(Shay Banon) #2

Sounds like a serialization problem... . Is there a chance that you can
create a simple test case and send it to me?

On Sat, Oct 9, 2010 at 3:00 AM, John Chang jchangkihtest2@gmail.com wrote:

Just upgraded from 0.9 to 0.11.0 and I find that
org.elasticsearch.search.SearchHit.fields() is returning null even when I
know the requested fields are there. When I do the query from the url
through curl (including the requested fields), they show up fine in the
returned json.

How I query:
response =

indexClient.search(Requests.searchRequest(getIndexName()).types(documentTypes).searchType(SearchType.QUERY_THEN_FETCH)

.source(SearchSourceBuilder.searchSource().query(QueryBuilders.customScoreQuery(QueryBuilders.queryString(query)).script("doc['"

  • sortField +

"'].value")).fields(fields).from(0).size(maxHitsPerIndexQuery).explain(true))

How I get the results:
List<Map<String, List>> results = new ArrayList<Map<String,
List>>();
for(SearchHit hit : response.hits().hits()) {

       Map<String, List> fieldMap = new HashMap<String, List>();

       for (Map.Entry<String, SearchHitField> field :

hit.fields().entrySet()) { //NPE here
fieldMap.put(field.getKey(), field.getValue().getValues());
}

       results.add(fieldMap);
   }

Note this is not a problem when I start local nodes in my unit tests; I
only
see this problem when I connect to a true stand-alone node started from the
command line.

Thanks.

View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/SearchHit-fields-returns-null-even-though-fields-are-there-tp1668529p1668529.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.


(John Chang) #3

Thanks for the offer, but I think I may have figured it out.

I simplified my client setup to the below snippet. I took out the index conifg stuff (as you advised in a different thread) because I was setting client(true). After doing this, the null fields problem went away. Does that make sense; would that explain it?

BTW - the problem was erratic. It generally gave me back null fields, but sometimes they where non-null but empty, and sometimes they were non-null and properly populated.

At any rate, since dropping the index config and simplifying the client setup to what you see below, it seems to work, at least for now.

private void initClient() {
    //elasticSearchHostsList should only ever be null or empty in unit test invocation
    if (elasticSearchHostsList != null && !elasticSearchHostsList.trim().equals("")) {
        NodeBuilder nb = NodeBuilder.nodeBuilder().client(true);
        nb.settings().put("discovery.zen.ping.unicast.hosts", elasticSearchHostsList);

        //cluster name must change as per environment to avoid s3 path clashes -- only in junit test invocation should it be left to default
        if (elasticSearchClusterName != null && !elasticSearchClusterName.trim().equals("")) {
            nb.settings().put("cluster.name", elasticSearchClusterName);
        }

        this.indexClient = nb.node().client();

        logger.info("Successfully initialized client with Elastic Search hosts " + elasticSearchHostsList);
    }

}

(Shay Banon) #4

Strange, this should not have fixed the problem ... . Is there a chance, if
you have time, for a recreation of this so I can delve into why this was
happening?

On Mon, Oct 11, 2010 at 9:13 AM, John Chang jchangkihtest2@gmail.comwrote:

Thanks for the offer, but I think I may have figured it out.

I simplified my client setup to the below snippet. I took out the index
conifg stuff (as you advised in a different thread) because I was setting
client(true). After doing this, the null fields problem went away. Does
that make sense; would that explain it?

BTW - the problem was erratic. It generally gave me back null fields, but
sometimes they where non-null but empty, and sometimes they were non-null
and properly populated.

At any rate, since dropping the index config and simplifying the client
setup to what you see below, it seems to work, at least for now.

private void initClient() {
//elasticSearchHostsList should only ever be null or empty in unit
test invocation
if (elasticSearchHostsList != null &&
!elasticSearchHostsList.trim().equals("")) {
NodeBuilder nb = NodeBuilder.nodeBuilder().client(true);
nb.settings().put("discovery.zen.ping.unicast.hosts",
elasticSearchHostsList);

       //cluster name must change as per environment to avoid s3 path

clashes -- only in junit test invocation should it be left to default
if (elasticSearchClusterName != null &&
!elasticSearchClusterName.trim().equals("")) {
nb.settings().put("cluster.name",
elasticSearchClusterName);
}

       this.indexClient = nb.node().client();

       logger.info("Successfully initialized client with Elastic

Search
hosts " + elasticSearchHostsList);
}

}

View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/SearchHit-fields-returns-null-even-though-fields-are-there-tp1668529p1678640.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.


(John Chang) #5

I'll try, but as I said, it is inconsistent. When I tried to write a little stand-alone test case to repro the problem, it worked fine. Of course, I can't ship you my whole project for numerous reasons. I'll try again to get a stand-alone repro test case to send you, but it may be elusive. THanks.


(Shay Banon) #6

Hey,

If you do manage to recreate it, it would be great. I will try and run
some tests on my end to see if I can reproduce this as well. If its a bug, I
would love to nail it.

-shay.banon

On Tue, Oct 12, 2010 at 1:24 AM, John Chang jchangkihtest2@gmail.comwrote:

I'll try, but as I said, it is inconsistent. When I tried to write a
little
stand-alone test case to repro the problem, it worked fine. Of course, I
can't ship you my whole project for numerous reasons. I'll try again to
get
a stand-alone repro test case to send you, but it may be elusive. THanks.

View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/SearchHit-fields-returns-null-even-though-fields-are-there-tp1668529p1684188.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.


(system) #7