# Maintaining @timestamp order of docs when using the reindex api

**URL:** <https://discuss.elastic.co/t/maintaining-timestamp-order-of-docs-when-using-the-reindex-api/351318>\
**Category:** Elasticsearch\
**Created:** [January 18, 2024, 6:41am UTC](https://discuss.elastic.co/t/maintaining-timestamp-order-of-docs-when-using-the-reindex-api/351318 "2024-01-18T06:41:11Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![petlit2049](https://avatars.discourse-cdn.com/v4/letter/p/e5b9ba/32.png) [@petlit2049](https://discuss.elastic.co/u/petlit2049)\
**Post date:** [January 18, 2024, 6:41am UTC](https://discuss.elastic.co/t/maintaining-timestamp-order-of-docs-when-using-the-reindex-api/351318/1 "2024-01-18T06:41:11Z")

</div>

There's a deprecated feature in the reindex api which makes (made?) it possible to maintain the timestamp order between docs being reindexed using the reindex api. However, the documentation states that this is a deprecated feature and should not be used and/or used with caution.

What I'd like to accomplish is to retrieve all documents in @timestamp order and reindex them in that order _but_ rewrite the @timestamp with an ingest pipeline. This seems(?) to work up to the version we're using currently using (v8.10.4) but it will be broken in the near future I assume... ☹

Is there a possibility to emulate this behavior using plain vanilla Elastisearch functionality _without_ relying on any external scroll-api based solutions? We have a number of use cases where we're not able to give users the ability to access ES using any programmatic (python, java whatever) means _except_ the standard ES api:s using the Kibana dev tool.

The way I've approached this so far is:

```auto
POST _reindex
{
  "source": {
    "index": "<some index>",
    "query": {
      "query_string": {
        "query": "@timestamp:*"
      }
    },
    "sort": [
        { "@timestamp" : {"order" : "asc"}}
    ]
  },
  "dest": {
    "index": "<some new index>",
    "pipeline": "set-timestamp"
  }
}

```

Is this a "proper" approach given my use case?

Many thanks in advance.

---

<div class="post-metadata">

**Author:** ![rtwolfe94022](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rtwolfe94022/32/132253_2.png) [@rtwolfe94022](https://discuss.elastic.co/u/rtwolfe94022)\
**Post date:** [January 26, 2024, 1:36am UTC](https://discuss.elastic.co/t/maintaining-timestamp-order-of-docs-when-using-the-reindex-api/351318/2 "2024-01-26T01:36:06Z")

</div>

Your approach of using the `_reindex` API with sorting by `@timestamp` in ascending order, combined with an ingest pipeline to rewrite the `@timestamp` , is quite effective for your objective of maintaining document order during reindexing in Elasticsearch. This method aligns well with your current constraints and version (v8.10.4). However, be mindful of potential performance impacts due to the resource-intensive nature of this process. Also, keep an eye on future Elasticsearch updates, as deprecated features may eventually be removed, possibly affecting this method. Testing and validation are crucial to ensure data integrity in the new index. While this approach works for now, consider alternative, more adaptable methods like scripting if your constraints change in the future. 👍📊

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 23, 2024, 1:36am UTC](https://discuss.elastic.co/t/maintaining-timestamp-order-of-docs-when-using-the-reindex-api/351318/3 "2024-02-23T01:36:11Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.