Whats the best way to migrate single node elasticsearch data to a multinode elasticsearch environment?

Elastic Stack Elasticsearch
1

I want to move my single node elasticsearch cluster data to a 3 node elasticsearch cluster. Only a few gb worth of data at first to test it out then proceed to move close to a 100gb. What is best way to this? Are any articles or instructions I can follow. Appreciate your help. Thank you.

2

Add more nodes to the existing one node cluster, then let the replicas allocate. That's about it really :slight_smile:

1 Like
3

As Mark said it is simple

I did this process,
I had three node cluster (old hardware)
got five newer systems. I added all five to existing cluster
and remove all three old node one by one.

Once you ready to remove node do this first, wait for while and check shard allocation untill no shard is left on system which you want to remove

PUT /_cluster/settings
{
"transient" :{
"cluster.routing.allocation.exclude._name" : "<hostname>"
 }
}

PUT /_cluster/settings
{
"transient" :{
"cluster.routing.allocation.exclude._ip" : "<ip of host>"
 }
}

GET /_cat/shards

1 Like
4

My single node cluster is an older version of elastic stack (kibana, logstash, elasticsearch). Its running in a production environment, so I don't want to change anything. I'd like to copy maybe a weeks worth of elasticsearch index to the new multinode elastic stack ( 3nodes elasticsearch, 1 node logstash, 1 node kibana) which will also be running in the same prod environment.

Would I just take the indexes from the single node elasticsearch and add it to just one of my new elasticsearch nodes then let it allocate across the other 2 nodes?

5

in that case look in to snapshot and restore. which is also very simple to do.
something like this

[https://qbox.io/blog/elasticsearch-data-snapshots-restore-tutorial]
(https://qbox.io/blog/elasticsearch-data-snapshots-restore-tutorial)

if you don't want that then create simple job which will read from old elastic cluster and write to new one.

Something like this if you need more control. like if you want to only ingest certain data. you can do that in your filter section.

input {
elasticsearch {
hosts => ["elkm01:9200"]
index => "old_index"
user => "elastic"
password => "xxxx"
 }
}

filter {}

output {
    elasticsearch {
        hosts => ["newmaster:9200"]
        index => "new_indexname"
       user => elastic
       password => xxxx
    }
  stdout { codec => rubydebug }
}
6

What version is the existing cluster, and what version do you want to go to?

8

Existing cluster: 5.6.16

New cluster: 7.6.1

9

Then use a remote reindex :slight_smile:

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.