Partial_fields: streaming or in-memory?

Hi ES'ers!

Just curious if anyone knows if partial_fields
http://www.elasticsearch.org/guide/reference/api/search/fields/uses a
streaming approach to filter _sourcehttp://www.elasticsearch.org/guide/reference/mapping/source-field/ documents.
The reason I ask is that we index some relatively large documents and if ES
can perform streaming filtering (as I believe Jackson supports) this would
be ideal. I couldn't really figure this out by browsing the source. It
looks as though this is implemented in-memory by pruning map keys. I could
be wrong :slight_smile:

Thanks again!

Bob

References:

[1] https://github.com/elasticsearch/elasticsearch/issues/1570
[2]
https://github.com/elasticsearch/elasticsearch/commit/cc3f44473fb7e0e543707403cea87e9b7ca7ad57

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Doing a bit more digging it looks as though this is the case:

[1] hitExecute
calls
[1] Object value = context.lookup().source().filter(field.includes(), field.
excludes());
calls
[2] loadSourceIfNeeded
calls
[2] reader.document(docId, sourceFieldVisitor);
calls
[3]

[1]


[2]

[3] http://lucene.apache.org/core/4_0_0/core/org/apache/lucene/index/IndexReader.html#document(int,
org.apache.lucene.index.StoredFieldVisitor)

On Monday, 29 April 2013 19:39:02 UTC-4, btiernay wrote:

Hi ES'ers!

Just curious if anyone knows if partial_fields
http://www.elasticsearch.org/guide/reference/api/search/fields/uses a
streaming approach to filter _sourcehttp://www.elasticsearch.org/guide/reference/mapping/source-field/ documents.
The reason I ask is that we index some relatively large documents and if ES
can perform streaming filtering (as I believe Jackson supports) this would
be ideal. I couldn't really figure this out by browsing the source. It
looks as though this is implemented in-memory by pruning map keys. I could
be wrong :slight_smile:

Thanks again!

Bob

References:

[1] https://github.com/elasticsearch/elasticsearch/issues/1570
[2]
https://github.com/elasticsearch/elasticsearch/commit/cc3f44473fb7e0e543707403cea87e9b7ca7ad57

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.