I have to migrate from ES5.6.3 to latest ES 7.x There are over 500 indices, 10 flavor of index, high number of index because of rollover ex index1-yyyy.mm and so on. The two options can be -
scripting - ex bash script connecting to new cluster
read all indexes from a file (this is done separately by connecting to 5.x cluster)
loop
create mapping
reindex from remote
Question- can index of same flavor be reindexed api in a single command with source and dest index as "index1-*"
same logic in a java program using high level REST client
Since the available servers to work on is Windows, I am having difficulty setting up bash script, If there is a way to do it in one shot for indexes of same flavor I would like that since it is a one time job.
Thanks @ylasri and @dadoonet . I tried reindex using wildcard with painless script. I have many indices Ex: audit-2018.01, audit-2018.02 and so on and want to create in another instance the indices by the same name. What's the correct way to reference remote index in the script?
POST _reindex
{
"conflicts": "proceed",
"source": {
"remote": {
"host": "http://localhost:9200"
},
"index": "audit*",
"size": 1000
},
"dest": {
"index": "audit",
"op_type": "create"
},
"script": {
"lang": "painless",
"source": "ctx._index = 'audit' + (remote._index.substring('audit'.length(), remote._index.length()))
}
}
@ylasri thank you, it does create multiple indexes. An interesting observation though (and my issue remains)
POST _reindex
{
"conflicts": "proceed",
"source": {
"remote": {
"host": "http://localhost:9200"
},
"index": "audit-2010.*",
"size": 100
},
"dest": {
"index": "audit-2010",
"op_type": "create"
},
"script": {
"lang": "painless",
"source": "ctx._index = 'audit-2010-' + (ctx._index.substring('audit-2010.'.length(), ctx._index.length()))"
}
}
creates multiple index audit-2010-01, audit-2010-02 but the below creates just one index . (the only difference is dot instead of dash since my original index are like audit-2010.01, audit-2010.01 I want to keep the same index name in the new instance
@yalsri it doesn't matter the selection by "index":"audit-2010*" . The question is how to escape the dot in painless. Escaping with \. gives error
"source": "ctx._index = 'audit-2010\.' + (ctx._index.substring('....
@dadoonet - David, this works in painless
"source": "ctx._index = (ctx._index.substring(1, ctx._index.length()))"
and this does not
"source": "ctx._index = (ctx._index.substring(0, ctx._index.length()))"
I do not get any syntax error but the default index name is created by reindex when using the later.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.