Migrating from ES 2.4 to 5.x

Elastic Stack Elasticsearch
1

I am trying to install the migration tool and that seemed to work fine. However, I cannot seem to connect to the host with localhost:9200/_plugin/elasticsearch-migration I get a 404 error, but ES is running. If I try to go into Kibana, I see these errors:

Courier Fetch Error: unhandled courier request error: Authorization Exception
Version: 4.5.4
Build: 10000
Error: unhandled courier request error: Authorization Exception
    at handleError (https://localhost/bundles/kibana.bundle.js?v=10000:80950:23)
    at DocRequest.AbstractReqProvider.AbstractReq.handleFailure (https://localhost/bundles/kibana.bundle.js?v=10000:80870:15)
    at https://localhost/bundles/kibana.bundle.js?v=10000:80764:18
    at Array.forEach (<anonymous>)
    at https://localhost/bundles/kibana.bundle.js?v=10000:80762:19
    at processQueue (https://localhost/bundles/commons.bundle.js?v=10000:42404:29)
    at https://localhost/bundles/commons.bundle.js?v=10000:42420:28
    at Scope.$eval (https://localhost/bundles/commons.bundle.js?v=10000:43648:29)
    at Scope.$digest (https://localhost/bundles/commons.bundle.js?v=10000:43459:32)
    at Scope.$apply (https://localhost/bundles/commons.bundle.js?v=10000:43756:25)

There are no errors in ES logs. Localhost replaced the ES server IP address. I am trying to do the remote check, so I had to add this to the yml file. For the allow-origin, I put the IP of the ES host. Is this correct?

http.cors.enabled:      true
http.cors.allow-origin: # the hostname or a regex which matches the hostname
2

I found an article saying to remove the http.cors entries. This allowed me to see the Kibana UI, but I still cannot seem to get the migration tool to come up at localhost:9200/_plugin/elasticsearch-migration

3

If you are running your cluster not on localhost did you try <ip_address>:9200/_plugin/elasticsearch-migration?

4

Yes, I get a 404 error when I try that. If I remove the /_plugin then it returns some data on Elasticsearch like the version, lucene version, etc.

5

Here is my Elasticsearch yml file. Maybe something is missing, but I don't think so. The only thing uncommented is the "network.host" and "https.cors.allow". Everything else in the file is commented out.

# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
 network.host: localhost
#
# Set a custom port for HTTP:
#
# http.port: 9200
#
# For more information, see the documentation at:
# <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html>
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
# discovery.zen.ping.unicast.hosts: ["host1", "host2"]
#
# Prevent the "split brain" by configuring the majority of nodes (total number of nodes / 2 + 1):
#
# discovery.zen.minimum_master_nodes: 3
#
# For more information, see the documentation at:
# <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-discovery.html>
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
# gateway.recover_after_nodes: 3
#
# For more information, see the documentation at:
# <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-gateway.html>
#
# ---------------------------------- Various -----------------------------------
#
# Disable starting multiple nodes on a single system:
#
# node.max_local_storage_nodes: 1
#
# Require explicit names when deleting indices:
#
# action.destructive_requires_name: true
 http.cors.enabled: true
# http.cors.allow-origin:
6

Hi,

Can you comment out/hash the two parameters you have uncommented. The network.host defaults to localhost. restart your server and then provide the output of http://localhost:9200/_cat

7

[root@localhost ~]# curl http://localhost:9200/_cat
=^.^=
/_cat/allocation
/_cat/shards
/_cat/shards/{index}
/_cat/master
/_cat/nodes
/_cat/indices
/_cat/indices/{index}
/_cat/segments
/_cat/segments/{index}
/_cat/count
/_cat/count/{index}
/_cat/recovery
/_cat/recovery/{index}
/_cat/health
/_cat/pending_tasks
/_cat/aliases
/_cat/aliases/{alias}
/_cat/thread_pool
/_cat/plugins
/_cat/fielddata
/_cat/fielddata/{fields}
/_cat/nodeattrs
/_cat/repositories
/_cat/snapshots/{repository}

8

What do you get for localhost:9200/_plugin/elasticsearch-migration?

9

[root@localhost ~]# curl http://localhost:9200/_plugin/elasticsearch-migration
<head><meta http-equiv="refresh" content="0; URL=/_plugin/elasticsearch-migration/"></head>

10

So you don't get no more HTTP 404. So you can continue with your migration?

11

Well it's a CentOS machine, so there is no GUI. When I try to access the server:9200/_plugin/elasticsearch-migration from my desktop, I get a 404. I need to access to GUI to do the things within it, like reindex and the various checks to ensure something won't break. I am just not sure where the issue is here :confused:

12

You need to get the centos machines ipv4 address ip a|grep inet, and set it for network.host ensure any firewall or iptables are turned off and restart elasticsearch to then access http://serverIPaddress:9200 via your desktop on the same LAN.

13

I had the server's IP set in the network.host line, but was asked to comment it out earlier. Maybe there was some miscommunication going on there. When I go to ip:9200, I was always able to see some data, but when I append /_plugin/elasticsearch-migration is when I get the 404.

IP4/6 tables are off and all firewall traffic is permitted internally. I can ssh to elasticsearch fine and Kibana works fine as well. Kibana is on a different VM on the same host and IP range.

This is what I get when i just use IP:9200

{
  "name" : "Fin",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "cPC9CNI6Si-pv-fh36VJgg",
  "version" : {
    "number" : "2.4.6",
    "build_hash" : "5376dca9f70f3abef96a77f4bb22720ace8240fd",
    "build_timestamp" : "2017-07-18T12:17:44Z",
    "build_snapshot" : false,
    "lucene_version" : "5.5.4"
  },
  "tagline" : "You Know, for Search"
}
14

Hello there. Has there been any update to this?

15

Hi,

You can try to tunnel ssh to access via browser. From your desktop machine you could try:

ssh -L 9999:localhost:9200 youruser@ipofyourcentos

Then in your desktop browser try http://localhost:9999

Maybe it helps

16

Thanks for the suggestion, but I am still getting a 404. Like I mentioned earlier, I can get some info if I use localhost:9200, but when I append /_plugin/elasticsearch-migration, I get a 404. Is there some step left out in the migration tool after this section?

Installation
To install, run the following command in your Elasticsearch home directory on a *nix system:

./bin/plugin install https://github.com/elastic/elasticsearch-migration/releases/download/v2.0.4/elasticsearch-migration-2.0.4.zip

The only thing listed after that is to open the browser.

I installed this in the directory /usr/share/elasticsearch where I successfully ran the above command. I just removed it and reinstalled it, but still no luck.

Thanks

17

Ok, so after I tried installing the migration plugin the second time and still couldn't get anything to work, I restarted elasticsearch and got the following error:

Starting elasticsearch: Exception in thread "main" java.lang.IllegalStateException: Could not load plugin descriptor for existing plugin [elasticsearch-migration]. Was the plugin built before 2.0?
Likely root cause: java.nio.file.AccessDeniedException: /usr/share/elasticsearch/plugins/elasticsearch-migration/plugin-descriptor.properties
        at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84)
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
        at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
        at java.nio.file.Files.newByteChannel(Files.java:361)
        at java.nio.file.Files.newByteChannel(Files.java:407)
        at java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:384)
        at java.nio.file.Files.newInputStream(Files.java:152)
        at org.elasticsearch.plugins.PluginInfo.readFromProperties(PluginInfo.java:87)
        at org.elasticsearch.plugins.PluginsService.getPluginBundles(PluginsService.java:378)
        at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:128)
        at org.elasticsearch.node.Node.<init>(Node.java:158)
        at org.elasticsearch.node.Node.<init>(Node.java:140)
        at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:143)
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:194)
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:286)
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:45)

I seen the access denied error so I tried to chmod 777 that file, but I'm still getting that error :confused:

18

So I fixed the permission issue along the way and now elasticsearch is starting, but I still cannot access elasticsearch-migration. I tried to edit /usr/share/elasticsearch/plugins/elasticsearch-migration/_site/index.html and substituted localhost with the server's IP, but still no luck lol

19

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.