# Logstash Elasticsearch Reindexing Question

**URL:** <https://discuss.elastic.co/t/logstash-elasticsearch-reindexing-question/22179>\
**Category:** Elasticsearch\
**Created:** [February 15, 2015, 8:54pm UTC](https://discuss.elastic.co/t/logstash-elasticsearch-reindexing-question/22179 "2015-02-15T20:54:33Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![sumithub](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sumithub/32/461_2.png) [@sumithub](https://discuss.elastic.co/u/sumithub)\
**Post date:** [February 15, 2015, 8:54pm UTC](https://discuss.elastic.co/t/logstash-elasticsearch-reindexing-question/22179/1 "2015-02-15T20:54:33Z")

</div>

I'm new to logstash and trying to make a proof of concept to reindex an  
already created Index (This index is not logs but standard files like  
.doc/pdf/emails etc). It will be greatly appreciated if you answer my  
question below and share your thoughts.

1. Can we reindex a non-log index by using logstash and keep the mappings  
of old index? If someone in this forum has tried it, please share your  
experience for the steps involved to achieve this.

My Observation:

By running the below configuration, elasticsearch is able to reindex but at  
the same time changes the mapping of new index. It also changed the index  
type to doc/text,html etc.

I think that's because elasticsearch parses the incoming JSON object from  
logstash message field via tika and determines the object type.

I know we could define our own indexing template and make mapping=strict  
but will that resolve the issue?

Here Input is any filesdata index created by elasticsearch and output is a  
new index. My logstash conf file looks like below

input {  
elasticsearch {  
host =\> "10.0.0.10"  
port =\> "9200"  
index =\> "filesdata"  
scroll =\> "1m"  
}  
}

output {  
elasticsearch {

```
             host => "10.0.0.11"
             protocol => "http"
             cluster => "node1"
             node_name => "indexer"
             index => "filesdata_15022015"
  }

```

stdout { codec =\> rubydebug }  
}

Looking forward for community experience in handing such scenario.

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/3e090d81-975a-4438-a355-dd04de4e772d%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/3e090d81-975a-4438-a355-dd04de4e772d%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [February 16, 2015, 7:50am UTC](https://discuss.elastic.co/t/logstash-elasticsearch-reindexing-question/22179/2 "2015-02-16T07:50:14Z")

</div>

Have you tried exporting and importing the mapping to the new cluster, then  
reindex using LS?

LS makes a number of assumptions when it creates mappings for a new index,  
which is probably what is happening.

On 16 February 2015 at 07:54, Sumit Arora [sumit101@gmail.com](mailto:sumit101@gmail.com) wrote:

> I'm new to logstash and trying to make a proof of concept to reindex an  
> already created Index (This index is not logs but standard files like  
> .doc/pdf/emails etc). It will be greatly appreciated if you answer my  
> question below and share your thoughts.
> 
> 1. Can we reindex a non-log index by using logstash and keep the mappings  
> of old index? If someone in this forum has tried it, please share your  
> experience for the steps involved to achieve this.
> 
> My Observation:
> 
> By running the below configuration, elasticsearch is able to reindex but  
> at the same time changes the mapping of new index. It also changed the  
> index type to doc/text,html etc.
> 
> I think that's because elasticsearch parses the incoming JSON object from  
> logstash message field via tika and determines the object type.
> 
> I know we could define our own indexing template and make mapping=strict  
> but will that resolve the issue?
> 
> Here Input is any filesdata index created by elasticsearch and output is a  
> new index. My logstash conf file looks like below
> 
> input {  
> elasticsearch {  
> host =\> "10.0.0.10"  
> port =\> "9200"  
> index =\> "filesdata"  
> scroll =\> "1m"  
> }  
> }
> 
> output {  
> elasticsearch {
> 
> ```
> host => "10.0.0.11"
> protocol => "http"
> cluster => "node1"
> node_name => "indexer"
> index => "filesdata_15022015"
> }
> 
> ```
> 
> stdout { codec =\> rubydebug }  
> }
> 
> Looking forward for community experience in handing such scenario.
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> To view this discussion on the web visit  
> [https://groups.google.com/d/msgid/elasticsearch/3e090d81-975a-4438-a355-dd04de4e772d%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/3e090d81-975a-4438-a355-dd04de4e772d%40googlegroups.com)  
> [https://groups.google.com/d/msgid/elasticsearch/3e090d81-975a-4438-a355-dd04de4e772d%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/3e090d81-975a-4438-a355-dd04de4e772d%40googlegroups.com?utm_medium=email&utm_source=footer)  
> .  
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X86fGzigQJ2j6fr%2B%3DF1dR%2B0Qh58jt\_-hELuL9KHHa7ekw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X86fGzigQJ2j6fr%2B%3DF1dR%2B0Qh58jt_-hELuL9KHHa7ekw%40mail.gmail.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![sumithub](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sumithub/32/461_2.png) [@sumithub](https://discuss.elastic.co/u/sumithub)\
**Post date:** [February 23, 2015, 10:29pm UTC](https://discuss.elastic.co/t/logstash-elasticsearch-reindexing-question/22179/3 "2015-02-23T22:29:49Z")

</div>

Thanks Mark for your reply. Yes I did look into the mapping and it works  
perfectly OK for simple documents. My mapping has nested and dynamic  
templates embedded and it's quite complex to implement re-indexing with  
logstash. I end up reindexing by using GIT  
plugin [GitHub - karussell/elasticsearch-reindex: Simple re-indexing. To backup, apply index settings changes and more ElasticMagic](https://github.com/karussell/elasticsearch-reindex) which worked well  
for my use case.

On Monday, February 16, 2015 at 7:54:34 AM UTC+11, Sumit Arora wrote:

> I'm new to logstash and trying to make a proof of concept to reindex an  
> already created Index (This index is not logs but standard files like  
> .doc/pdf/emails etc). It will be greatly appreciated if you answer my  
> question below and share your thoughts.
> 
> 1. Can we reindex a non-log index by using logstash and keep the mappings  
> of old index? If someone in this forum has tried it, please share your  
> experience for the steps involved to achieve this.
> 
> My Observation:
> 
> By running the below configuration, elasticsearch is able to reindex but  
> at the same time changes the mapping of new index. It also changed the  
> index type to doc/text,html etc.
> 
> I think that's because elasticsearch parses the incoming JSON object from  
> logstash message field via tika and determines the object type.
> 
> I know we could define our own indexing template and make mapping=strict  
> but will that resolve the issue?
> 
> Here Input is any filesdata index created by elasticsearch and output is a  
> new index. My logstash conf file looks like below
> 
> input {  
> elasticsearch {  
> host =\> "10.0.0.10"  
> port =\> "9200"  
> index =\> "filesdata"  
> scroll =\> "1m"  
> }  
> }
> 
> output {  
> elasticsearch {
> 
> ```
> host => "10.0.0.11"
> protocol => "http"
> cluster => "node1"
> node_name => "indexer"
> index => "filesdata_15022015"
> }
> 
> ```
> 
> stdout { codec =\> rubydebug }  
> }
> 
> Looking forward for community experience in handing such scenario.

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/f49dac20-da0c-4993-bf3b-829379a0134c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f49dac20-da0c-4993-bf3b-829379a0134c%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 12:30am UTC](https://discuss.elastic.co/t/logstash-elasticsearch-reindexing-question/22179/4 "2017-07-06T00:30:39Z")

</div>