Reindex from remote (AWS Service)

Elastic Stack Elasticsearch
1

I'm running a Amazon Elasticsearch Service 5.1 (last version available there).
I'd like to reindex my documents from a remote server, but I can't do this on AWS because I can't modify the elasticsearch.yml file.
AWS Elastic Service doesn't provide a way to put reindex.remote.whitelist flag in his config.

What should I do?

Thanks

2

Unfortunately, I'm afraid that's a rather strict requirement of the _reindex API. You will not be able to use _reindex to accomplish this task.

You can, however, create your own script to pull from one source using the scroll API, and then send all the documents found to the target cluster. It would be a rather manual process, I'm afraid.

3

Can you try with this API?

https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-update-settings.html

4

Yes @dadoonet, but it doesn't work.

There is a topic about this where a contributor explains why it shouldn't work:

5

@dadoonet, the documentation says it must be in the elasticsearch.yml file:

Remote hosts have to be explicitly whitelisted in elasticsearch.yaml using the reindex.remote.whitelist property. It can be set to a comma delimited list of allowed remote host and port combinations (e.g. otherhost:9200, another:9200, 127.0.10.*:9200, localhost:*). Scheme is ignored by the whitelist - only host and port are used.

I think that's by design, that it must be hard-coded before startup.

6

Thanks for suggestion, but I was trying to avoid that. :disappointed:

7

Of course! I was dumb when I thought about that. Totally insecure proposal but thankfully rejected by elasticsearch :slight_smile:

I wonder if #cloud supports defining this in elasticsearch.yml file BTW.

8

It does. Scroll down to the bottom of the page here to see it listed, specifically.

1 Like
9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.