Persisting Aggregations

I've not yet used the aggregations framework, but one question that has
come up recently with contacts and prospective clients is how best to
persist aggregations in ElasticSearch for repeated use.

If I have understood the documentation correctly, the aggregation framework
does a pretty good job of using shard caching to make repeated-or-similar
queries as efficient as possible, but it would - presumably - be even
better if "static" results (i.e. which will hardly ever - or never -
change) could be persisted in some way (in a dedicated index, for example).

Is this possible "internally" (i.e. to GET an aggregation result and POST
it in one call) or would one simply have to extract the desired data and
then post it oneself?

Regards, Andrew

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/2b492a47-1fa6-40f1-a14e-54ccb7fe2a0e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

The Aggs Fw doesn't allow for persisting results, mainly because it is
targeted at real-time data that can still change, but it does support
caching as of 1.4. That is, if you issue the same query & aggregations
request again and again you will be served directly from cache, given the
data hasn't changed.

That is to say, if you care about performance, the caching layer should be
the answer. If you need other things (point in time view of data, further
processing, etc) you will need to store the results back to ES or other
storage as a document.

HTH

--

Itamar Syn-Hershko
http://code972.com | @synhershko https://twitter.com/synhershko
Freelance Developer & Consultant
Lucene.NET committer and PMC member

On Tue, Feb 3, 2015 at 11:21 AM, AndrewK kenworthyas@gmail.com wrote:

I've not yet used the aggregations framework, but one question that has
come up recently with contacts and prospective clients is how best to
persist aggregations in Elasticsearch for repeated use.

If I have understood the documentation correctly, the aggregation
framework does a pretty good job of using shard caching to make
repeated-or-similar queries as efficient as possible, but it would -
presumably - be even better if "static" results (i.e. which will hardly
ever - or never - change) could be persisted in some way (in a dedicated
index, for example).

Is this possible "internally" (i.e. to GET an aggregation result and POST
it in one call) or would one simply have to extract the desired data and
then post it oneself?

Regards, Andrew

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/2b492a47-1fa6-40f1-a14e-54ccb7fe2a0e%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/2b492a47-1fa6-40f1-a14e-54ccb7fe2a0e%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAHTr4Zs6c4tbG-2vXYowbpcA45MTQty1i6Hquv%3DOYVYOWSp9%3DQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Thanks for the feedback Itamar: I had a feeling that that would be the case
but the confirmation is helpful (and storing the results back in
ES/elsewhere is not a problem).

Regards, Andrew

Am Dienstag, 3. Februar 2015 10:52:33 UTC+1 schrieb Itamar Syn-Hershko:

The Aggs Fw doesn't allow for persisting results, mainly because it is
targeted at real-time data that can still change, but it does support
caching as of 1.4. That is, if you issue the same query & aggregations
request again and again you will be served directly from cache, given the
data hasn't changed.

That is to say, if you care about performance, the caching layer should be
the answer. If you need other things (point in time view of data, further
processing, etc) you will need to store the results back to ES or other
storage as a document.

HTH

--

Itamar Syn-Hershko
http://code972.com | @synhershko https://twitter.com/synhershko
Freelance Developer & Consultant
Lucene.NET committer and PMC member

On Tue, Feb 3, 2015 at 11:21 AM, AndrewK <kenwo...@gmail.com <javascript:>

wrote:

I've not yet used the aggregations framework, but one question that has
come up recently with contacts and prospective clients is how best to
persist aggregations in Elasticsearch for repeated use.

If I have understood the documentation correctly, the aggregation
framework does a pretty good job of using shard caching to make
repeated-or-similar queries as efficient as possible, but it would -
presumably - be even better if "static" results (i.e. which will hardly
ever - or never - change) could be persisted in some way (in a dedicated
index, for example).

Is this possible "internally" (i.e. to GET an aggregation result and POST
it in one call) or would one simply have to extract the desired data and
then post it oneself?

Regards, Andrew

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/2b492a47-1fa6-40f1-a14e-54ccb7fe2a0e%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/2b492a47-1fa6-40f1-a14e-54ccb7fe2a0e%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5d883aef-07d7-4ad1-82c4-472822cab28f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.