Querying arrays of objects with optional fields within a record

Hi,

I'm having an issue that I'm hoping someone can help me with.

The basic problem is that if there is an object array structure in the data where some fields in the object are optional, I can't reconstruct the original objects after querying the data because of the way results are returned. I hope I'm just missing something.

I'll put together a simple contrived example that demonstrates the point.

There is a simple object like this with two fields:
{ "field1" : "a", "field2" : "b" }

And a "data" field that is an array of these.

{ "data" : [ { "field1" : "a", "field2" : "b" }, { "field1" : "c" }, { "field2" : "d" } ] }

Sometimes either field1 or field 2 don't have a value.

And the data array is itself just a field within a record.

Now when querying this data, you get arrays of results for each field, looking something like this for each record "hit":

{ "data.field1" : ["a", "c"], "data.field2" : ["b", "d"] }

Within the record "hit", the original objects are completely lost, and there is no way to reconstruct them.

The results imply that the objects are like this:

{ "data" : [ { "field1" : "a", "field2" : "b" }, { "field1" : "c" , "field2" : "d" } ] }

When they are actually this:

{ "data" : [ { "field1" : "a", "field2" : "b" }, { "field1" : "c" }, { "field2" : "d" } ] }

I should also mention that I'm using the Java client.

Also, I'm specifiying the fields in the query, like this in the example given above:
"fields" : [ "data.field1", "data.field2"]

Specifying no fields only returns the metadata (id, type, score), not the field data.

Also, specifying a non-leaf path like this returns no hits:

"fields" : [ "data" ]

So I found out about nested object mappings, and it looks like they will solve this problem.