Shelef
December 22, 2014, 12:28pm
1
Fields are used to pull data from stored fields whereas source filtering is
--
Itamar Syn-Hershkohttp://code972.com | @synhershko https://twitter.com/synhershko http://manning.com/synhershko/
On Mon, Dec 22, 2014 at 2:28 PM, Shelef shlaflaf@gmail.com wrote:
I read about two ways to filter the fields returned by elasticsearch.http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-fields.html http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-source-filtering.html .
--elasticsearch+unsubscribe@googlegroups.com .https://groups.google.com/d/msgid/elasticsearch/0a8593ca-d17f-4f6d-b3b8-b5ee10196892%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/0a8593ca-d17f-4f6d-b3b8-b5ee10196892%40googlegroups.com?utm_medium=email&utm_source=footer https://groups.google.com/d/optout .
--elasticsearch+unsubscribe@googlegroups.com .https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZtVDGbnuBdGgxHavr3O9ToO%2BC_5Tg%3DE8tntw_5yW%3Djm%3Dg%40mail.gmail.com .https://groups.google.com/d/optout .
nik9000
December 22, 2014, 12:40pm
3
General rule:
Use source filtering unless you can't. Source filtering works if theneed to store fields if they are synthetic
Advanced thing I've never used:
Its possible to get some performance gains by setting a small field to
Nikshlaflaf@gmail.com wrote:
I read about two ways to filter the fields returned by elasticsearch.http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-fields.html http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-source-filtering.html .
--elasticsearch+unsubscribe@googlegroups.com .https://groups.google.com/d/msgid/elasticsearch/0a8593ca-d17f-4f6d-b3b8-b5ee10196892%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/0a8593ca-d17f-4f6d-b3b8-b5ee10196892%40googlegroups.com?utm_medium=email&utm_source=footer https://groups.google.com/d/optout .
--elasticsearch+unsubscribe@googlegroups.com .https://groups.google.com/d/msgid/elasticsearch/CAPmjWd0yMv9%3DdjpAOUQ5P%3D0dB1GN8DATghhk2zZgsk%3De14Y8jA%40mail.gmail.com .https://groups.google.com/d/optout .
nik9000
December 22, 2014, 12:46pm
4
Does source fallback? I remember trying and getting nothing.itamar@code972.com wrote:
Fields are used to pull data from stored fields whereas source filtering
--
Itamar Syn-Hershkohttp://code972.com | @synhershko https://twitter.com/synhershko http://manning.com/synhershko/
On Mon, Dec 22, 2014 at 2:28 PM, Shelef shlaflaf@gmail.com wrote:
I read about two ways to filter the fields returned by elasticsearch.http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-fields.html http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-source-filtering.html .
--elasticsearch+unsubscribe@googlegroups.com .https://groups.google.com/d/msgid/elasticsearch/0a8593ca-d17f-4f6d-b3b8-b5ee10196892%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/0a8593ca-d17f-4f6d-b3b8-b5ee10196892%40googlegroups.com?utm_medium=email&utm_source=footer https://groups.google.com/d/optout .
--elasticsearch+unsubscribe@googlegroups.com .https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZtVDGbnuBdGgxHavr3O9ToO%2BC_5Tg%3DE8tntw_5yW%3Djm%3Dg%40mail.gmail.com https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZtVDGbnuBdGgxHavr3O9ToO%2BC_5Tg%3DE8tntw_5yW%3Djm%3Dg%40mail.gmail.com?utm_medium=email&utm_source=footer https://groups.google.com/d/optout .
--elasticsearch+unsubscribe@googlegroups.com .https://groups.google.com/d/msgid/elasticsearch/CAPmjWd3Q1uf86aPqy4uo9cbpOerPKDONViCwPog6AT7nwiEHJg%40mail.gmail.com .https://groups.google.com/d/optout .
From the docs:
By default operations return the contents of the _source field unless you
Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today.
--
Itamar Syn-Hershkohttp://code972.com | @synhershko https://twitter.com/synhershko http://manning.com/synhershko/
On Mon, Dec 22, 2014 at 2:46 PM, Nikolas Everett nik9000@gmail.com wrote:
Does source fallback? I remember trying and getting nothing.itamar@code972.com wrote:
Fields are used to pull data from stored fields whereas source filtering
--
Itamar Syn-Hershkohttp://code972.com | @synhershko https://twitter.com/synhershko http://manning.com/synhershko/
On Mon, Dec 22, 2014 at 2:28 PM, Shelef shlaflaf@gmail.com wrote:
I read about two ways to filter the fields returned by elasticsearch.http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-fields.html http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-source-filtering.html .
--elasticsearch+unsubscribe@googlegroups.com .https://groups.google.com/d/msgid/elasticsearch/0a8593ca-d17f-4f6d-b3b8-b5ee10196892%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/0a8593ca-d17f-4f6d-b3b8-b5ee10196892%40googlegroups.com?utm_medium=email&utm_source=footer https://groups.google.com/d/optout .
--elasticsearch+unsubscribe@googlegroups.com .https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZtVDGbnuBdGgxHavr3O9ToO%2BC_5Tg%3DE8tntw_5yW%3Djm%3Dg%40mail.gmail.com https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZtVDGbnuBdGgxHavr3O9ToO%2BC_5Tg%3DE8tntw_5yW%3Djm%3Dg%40mail.gmail.com?utm_medium=email&utm_source=footer https://groups.google.com/d/optout .
--elasticsearch+unsubscribe@googlegroups.com .https://groups.google.com/d/msgid/elasticsearch/CAPmjWd3Q1uf86aPqy4uo9cbpOerPKDONViCwPog6AT7nwiEHJg%40mail.gmail.com https://groups.google.com/d/msgid/elasticsearch/CAPmjWd3Q1uf86aPqy4uo9cbpOerPKDONViCwPog6AT7nwiEHJg%40mail.gmail.com?utm_medium=email&utm_source=footer https://groups.google.com/d/optout .
--elasticsearch+unsubscribe@googlegroups.com .https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZscB0wvnAGN0%2B6GOF%2BCSV7u-ziH4ybBfpj%3DUa-c7NNUzg%40mail.gmail.com .https://groups.google.com/d/optout .
jpountz
December 24, 2014, 4:45pm
6
On Mon, Dec 22, 2014 at 1:33 PM, Itamar Syn-Hershko itamar@code972.com
Fields are used to pull data from stored fields whereas source filtering
I don't think we can deprecate fields because of synthetic fields_source filtering has the drawback that you might need
opened 04:53PM - 22 Dec 14 UTC
closed 03:56PM - 24 May 22 UTC
>enhancement
high hanging fruit
:Search/Mapping
Team:Search
Today we store the `_source` is a single big binary stored field. While this is … great for simplicity, this also has the bad side-effect to encourage to store fields individually in order to save some json parsing when there are a couple of large field values and we are only interested in some short values. Maybe we could try to be a bit smarter and store the `_source` across several stored fields so that it would not be an issue anymore?
Random idea: given a document that looks like:
``` json
{
"title": "short_string",
"body": "very_very_very_very_long_string",
"array": [2, 3, 10],
"foo": {
"foo": 42,
"bar": "baz"
}
}
```
we could for instance store all the top-level fields into their own stored field
| Field | Values |
| --- | --- |
| title | "short_string" |
| body | "very_very_very_very_long_string" |
| array | [2, 3, 10] |
| foo | {"foo": 42, "bar": "baz"} |
or maybe even each value individually (but it becomes more complicated with arrays of objects):
| Field | Values |
| --- | --- |
| title | "short_string" |
| body | "very_very_very_very_long_string" |
| array | [2, 3, 10] |
| foo.foo | 42 |
| foo.bar | "baz" |
Then we would have to make `_source` filtering aware of the way fields are stored, and for instance if we store only top-level fields into their own stored field then we could translate an include rule like `foo.*` to "retrieve field `foo`", and `foo.bar.*` to "get everything under `bar` for field `foo`".
--
--elasticsearch+unsubscribe@googlegroups.com .https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j54F4_uQTS_K39r3g_tAuY%3DHfMGhh%2BFeMwHDo00Zytj5Q%40mail.gmail.com .https://groups.google.com/d/optout .