# Is there a way to reindex data in ES from start to now()?

**URL:** <https://discuss.elastic.co/t/is-there-a-way-to-reindex-data-in-es-from-start-to-now/219830>\
**Category:** Elasticsearch\
**Created:** [February 18, 2020, 4:07pm UTC](https://discuss.elastic.co/t/is-there-a-way-to-reindex-data-in-es-from-start-to-now/219830 "2020-02-18T16:07:14Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![fallenreaper](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fallenreaper/32/62407_2.png) [@fallenreaper](https://discuss.elastic.co/u/fallenreaper)\
**Post date:** [February 18, 2020, 4:07pm UTC](https://discuss.elastic.co/t/is-there-a-way-to-reindex-data-in-es-from-start-to-now/219830/1 "2020-02-18T16:07:14Z")

</div>

I figured there was a built in way to say: Hey Master node, reindex on propertyA, propertyB, propertyC for independent faster lookups.

My ES DB is about 250g large now on a single dual purpose Master/Data node. I was having issues where kibana queries were taking \> 30 seconds, so it sounds like it might be time to start optimizing.

While I can tweak my logstash instances to pass tweaked data, I was not sure if it is possible to have elastic go over the currently existing data and reindex by more keywords etc.

I can likely update my filter groks in logstash

```auto
filter {
	grok {
		match => ["path", "%{GREEDYDATA}/%{GREEDYDATA:filename}\.txt"]
	}
	grok {
		match => {
			"message" => "%{DATA:sampleinfo}[:;]%{GREEDYDATA:backupinfo}"
		}
	}
	mutate {
		gsub => ["backupinfo", "[\n\r\t]", ""]
	}
}

```

But I wasnt sure if I can do this from within Elasticsearch.

Something like: Starting now(), reindex all X,Y,Z and turn it into A,B,C. I figured that when all logstash are updated they will start ingesting the correct information. So i would just need to do an update for all documents from: Oldest entry to now()

I will make a follow up post in Logstash on how to update logstash the things which need to be indexed for the fastest lookup are: timestamp, filename, sampleinfo, backupinfo

I presume there way was a way to redefine a property to have a different value for indexing. All the data are just variable character strings,

---

<div class="post-metadata">

**Author:** ![fallenreaper](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fallenreaper/32/62407_2.png) [@fallenreaper](https://discuss.elastic.co/u/fallenreaper)\
**Post date:** [February 18, 2020, 11:15pm UTC](https://discuss.elastic.co/t/is-there-a-way-to-reindex-data-in-es-from-start-to-now/219830/2 "2020-02-18T23:15:52Z")

</div>

Can I execute a command in Elasticsearch to reindex everything while also ingesting new Data? I was not sure if i could execute a function which will reindex a property from X type to Y type? I presume there is a way I can define the properties (sampleinfo and backupinfo and the timestamp bucketing) which are stored and reindex them to something else for easier lookups?

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [February 19, 2020, 2:27pm UTC](https://discuss.elastic.co/t/is-there-a-way-to-reindex-data-in-es-from-start-to-now/219830/3 "2020-02-19T14:27:15Z")

</div>

Take a look at the [reindex API](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html), which can be run while new data is being indexed.

---

<div class="post-metadata">

**Author:** ![fallenreaper](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fallenreaper/32/62407_2.png) [@fallenreaper](https://discuss.elastic.co/u/fallenreaper)\
**Post date:** [February 19, 2020, 4:38pm UTC](https://discuss.elastic.co/t/is-there-a-way-to-reindex-data-in-es-from-start-to-now/219830/4 "2020-02-19T16:38:27Z")

</div>

I might be using the wrong words here. It seems that the API is like moving data from Cluster "foo" to a new cluster "bar" I think? I think this will be useful later for sure.

I was thinking that the way some of my variables being defined may be why my when querying, the resultset is really slow to fetch. I was thinking there may be a way to update the property type by doing some command such that when querying, it would find results faster?

Im not sure how ES handles typings, since I want to streamline the ES instance Specifically around the config variables: timestamp (built in), filename, sampleinfo, backupinfo

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [February 20, 2020, 7:42am UTC](https://discuss.elastic.co/t/is-there-a-way-to-reindex-data-in-es-from-start-to-now/219830/5 "2020-02-20T07:42:23Z")

</div>

Hey,

reindex from remote is only. a part of reindex, you can also specify another index within your cluster.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 19, 2020, 7:42am UTC](https://discuss.elastic.co/t/is-there-a-way-to-reindex-data-in-es-from-start-to-now/219830/6 "2020-03-19T07:42:23Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.