Hello!
We're trying to implement a has_parent filter, and we're running into
a strange problem where this query works:
{
"filter": {
"has_parent": {
"query": {
"term": {
"sId": "095520ef2aaa5d1e14d46296fdb2464f9b21dcaf"
}
},
"type": "changeset"
}
},
"query": {
"term": {
"ixRepo": 330
}
}
}
but this query does not:
{
"filter": {
"has_parent": {
"query": {
"term": {
"ixHostedDB": -1
}
},
"type": "changeset"
}
},
"query": {
"term": {
"ixRepo": 330
}
}
}
First, let me explain how the filter works a little: Using the above
query as an example, it should return all documents that not only have
a parent type of "changeset" but also match the inner query. So the
first query works fine: it returns a bunch of documents with a
changeset parent that has sId=0955[snip].
However, the second query returns zero documents even though we know
there are successful matches in the index, i.e. documents that
definitely have a changeset parent with ixHostedDB definitely set to
-1. As far as we can tell, the two fields (sId and ixHostedDB) only
differ by their types: sId is mapped as a string and ixHostedDB is
mapped as an integer. Here's our mapping:
{
"kiln-2-8" : {
"changesetrepo" : {
"_source" : {
"enabled" : false
},
"_routing" : {
"required" : true
},
"properties" : {
"ixRepo" : {
"type" : "integer"
}
},
"_parent" : {
"type" : "changeset"
}
}
"changeset" : {
"properties" : {
"id" : {
"type" : "string"
},
"dt" : {
"format" : "dateOptionalTime",
"type" : "date"
},
"sDescription" : {
"store" : "yes",
"analyzer" : "snowball",
"term_vector" : "with_positions_offsets",
"type" : "string"
},
"ixHostedDB" : {
"type" : "integer"
},
"sParentId1" : {
"type" : "string"
},
"sParentId2" : {
"type" : "string"
},
"sBranch" : {
"type" : "string"
},
"sId" : {
"type" : "string"
},
"sExtra" : {
"type" : "string"
},
"sAuthor" : {
"type" : "string"
}
}
}
}
}
And here is our code for the has_parent filter, plus some basic tests,
squashed into one commit:
https://github.com/hlian/elasticsearch/commit/8f9bbaa0897678e7f2e82d1e90b23f3add20e164
The code's still incomplete (we'd like to implement the Query
equivalents as well and clean up the tests), but if you guys could
take a look at it and see if you spot any bugs, that'd be great.
We've tried stepping through the ES and Lucene source code to no
avail. The one interesting thing we found is that, inside
TermQuery#TermWeight's constructor in org.apache.lucene.search,
"r.docFreq(term)" always returns 0 for the failing query, where term
corresponds to "ixHostedDB:-1". Whereas it returns positive values for
the successful query, i.e. when term corresponds to "sId:0955[snip]"
or "sAuthor:hao" or basically anything except ixHostedDB. We think
that the docFreq being zero is related to our lack of results, but
again we're new to both ES and Lucene's code so we're not sure.
It's additionally strange because it passes our unit tests, which
tests among other things a simplified example of the query that isn't
working (the one with ixHostedDB).
Anyway, any feedback would be appreciated!
Thanks,
Hao.