# Scripting with multiple documents **URL:** \ **Category:** Elasticsearch\ **Created:** [August 7, 2015, 12:42pm UTC](https://discuss.elastic.co/t/scripting-with-multiple-documents/27044 "2015-08-07T12:42:44Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![oravecz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/oravecz/32/44822_2.png) [@oravecz](https://discuss.elastic.co/u/oravecz)\ **Post date:** [August 7, 2015, 12:42pm UTC](https://discuss.elastic.co/t/scripting-with-multiple-documents/27044/1 "2015-08-07T12:42:44Z")
All of the scripting examples (whether sorting, filtering or computing new values) take place on a single document. Is it possible to remove filtered results prior to aggregation using custom scripts or a plugin? Use case: My documents are idempotent, and represent changes to a user's account. Each account update indexes a new document. I want to aggregate records on an account field, but I only want to include the user's latest update in the aggregation. ---
**Author:** ![nik9000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik9000/32/44947_2.png) [@nik9000](https://discuss.elastic.co/u/nik9000)\ **Post date:** [August 7, 2015, 12:55pm UTC](https://discuss.elastic.co/t/scripting-with-multiple-documents/27044/2 "2015-08-07T12:55:08Z")
> [@oravecz](#): > > Use case: My documents are idempotent, and represent changes to a user's account. Each account update indexes a new document. I want to aggregate records on an account field, but I only want to include the user's latest update in the aggregation. I'd have an index that just contains the latest data per user in that case. If you still need the history then maybe two indexes? It'd be a pain to make sure they stay in sync but it'd be simpler than trying to squash the documents on the fly during an aggregation. ---
**Author:** ![oravecz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/oravecz/32/44822_2.png) [@oravecz](https://discuss.elastic.co/u/oravecz)\ **Post date:** [August 7, 2015, 1:57pm UTC](https://discuss.elastic.co/t/scripting-with-multiple-documents/27044/3 "2015-08-07T13:57:43Z")
Are you suggesting that before I index the latest change into one index, I copy the existing record to an "archived" index and delete it from the "live" index? I don't get it. More to the point, are you suggesting this approach because there is no way to filter an aggregated collection of records to include only the latest record? What I'm trying to do is similar to this pseudo-SQL statement: SELECT \* from T where T.submitted = ( SELECT MAX(T.submitted) WHERE T.user = ? ) ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [August 7, 2015, 2:17pm UTC](https://discuss.elastic.co/t/scripting-with-multiple-documents/27044/4 "2015-08-07T14:17:01Z")
I think he is suggesting keeping all the raw records in one index and have a separate index where you store the latest status, using the user ID as a key so that any update for a user coming in will overwrite the previous status. That way you have fast access to the last change made for every user as wells the entire history. ---
**Author:** ![oravecz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/oravecz/32/44822_2.png) [@oravecz](https://discuss.elastic.co/u/oravecz)\ **Post date:** [August 7, 2015, 3:27pm UTC](https://discuss.elastic.co/t/scripting-with-multiple-documents/27044/5 "2015-08-07T15:27:37Z")
@Christian_Dahlqvist Thanks, that makes more sense. 😄 However, that is not an option for me due to the size of these indices and our rollover strategy would get really complicated, but I do see how it is a workaround. So, no ability to create an aggregation that limits the records to only include the latest record? Something like max(\_timestamp)? No way to do it with a custom plugin? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [August 7, 2015, 4:08pm UTC](https://discuss.elastic.co/t/scripting-with-multiple-documents/27044/6 "2015-08-07T16:08:20Z")
Although I am not sure how you would do it, it may be possible to do all that processing at query time. This could however be slow and may also be difficult to scale well. Using a separate index to hold the most recent state will make these queries much more efficient as well as scale well. If you are performing this type of query rarely, doing all this work for each query may be fine, but if you need to get this information on a regular basis, you will most likely benefit from doing the work up front by having a separate index. ---
**Author:** ![oravecz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/oravecz/32/44822_2.png) [@oravecz](https://discuss.elastic.co/u/oravecz)\ **Post date:** [August 12, 2015, 12:45pm UTC](https://discuss.elastic.co/t/scripting-with-multiple-documents/27044/7 "2015-08-12T12:45:02Z")
Thanks. If I cannot filter a group of records in a custom plugin, then I will have to post process my results programmatically. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 11:56pm UTC](https://discuss.elastic.co/t/scripting-with-multiple-documents/27044/8 "2017-07-05T23:56:12Z")