HELP ! Upgrade ES 6.7.0 to 7.1.0

skyluke.1987 · June 12, 2019, 3:44am

Hi all, I need some help on the upgrading. Following the guide from here: https://www.elastic.co/blog/getting-started-with-elasticsearch-security.

I downloaded the installation files and configured. When about to start, it throw me the following error.

305 main ERROR No Log4j 2 configuration file found. Using default configuration
(logging only errors to the console), or user programmatically provided configurations.
Set system property 'log4j2.debug' to show Log4j 2 internal initialization logging.
See https://logging.apache.org/log4j/2.x/manual/configuration.html for instructions on how to
configure Log4j 2

SettingsException[Failed to load settings from /etc/elasticsearch-master/config/elasticsearch.yml];
nested: AccessDeniedException[/etc/elasticsearch-master/config/elasticsearch.yml];

drwxr-xr-x 2 root root config
-rw-rw---- 1 root root elasticsearch.yml

Please advise.

dadoonet · June 12, 2019, 5:53am

Elasticsearch does not run as root so you probably need to change the owner to elasticsearch.

skyluke.1987 · June 12, 2019, 6:23am

Ho Dadoonet, do I need to change all the folders in elasticsearch ownership to "elasticsearch" ?

dadoonet · June 12, 2019, 6:38am

Probably.

I'm just wondering how did you install it in the first place.

skyluke.1987 · June 12, 2019, 6:39am

I was following this guide. Download the package and unzipped it. Then modify the configuration file and start the service from bin folder.

skyluke.1987 · June 12, 2019, 8:55am

Hi all, files under /var/log/elasticsearch, these files are "Permission Denied" why is it so ??

skyluke.1987 · June 13, 2019, 1:38am

Dear all, I notice there is one step to delete all (or reindex) the indexes before loading the new ES.

Should I delete them all? and start fresh? How do I do that?

skyluke.1987 · June 13, 2019, 8:45am

Elasticsearch 7.1.0. May I know what is the above? Do I need to change it?

skyluke.1987 · June 13, 2019, 9:01am

Hi all, may I know what error is this about?

dadoonet · June 13, 2019, 9:12am

Please don't post images of text as they are hardly readable and not searchable.

Instead paste the text and format it with </> icon. Check the preview window.

Please edit your posts.

skyluke.1987 · June 13, 2019, 9:37am

{"type": "server", "timestamp": "2019-06-13T17:34:40,193+0800", "level": "WARN", "component": "o.e.c.c.ClusterFormationFailureHelper", "cluster.name": "wazuh-es", "node.name": "eta10", "message": "master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [10.0.106.144, node-2] to bootstrap a cluster: have discovered []; discovery will continue using [10.0.106.143:9300] from hosts providers and [{eta10}{qDMaRgdWS92S_nyV3tQTJA}{-LH5RdZWQOSiOdnr2H-TUw}{10.0.106.144}{10.0.106.144:9300}{ml.machine_memory=270444232704, xpack.installed=true, ml.max_open_jobs=20}] from last-known cluster state; node term 0, last-accepted version 0 in term 0" }

Hi all, I notice the "wazuh-es_server.json" file prompting this warning msg. What does this means ?

skyluke.1987 · June 14, 2019, 1:31am

Java HotSpot(TM) 64-Bit Server VM warning: Cannot open file logs/gc.log due to Permission denied

Exception in thread "main" org.elasticsearch.bootstrap.BootstrapException: java.nio.file.AccessDeniedException: /etc/elasticsearch/config/elasticsearch.keystore
Likely root cause: java.nio.file.AccessDeniedException: /etc/elasticsearch/config/elasticsearch.keystore
    at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84)
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
    at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
    at java.nio.file.Files.newByteChannel(Files.java:361)
    at java.nio.file.Files.newByteChannel(Files.java:407)
    at org.apache.lucene.store.SimpleFSDirectory.openInput(SimpleFSDirectory.java:77)
    at org.elasticsearch.common.settings.KeyStoreWrapper.load(KeyStoreWrapper.java:206)
    at org.elasticsearch.bootstrap.Bootstrap.loadSecureSettings(Bootstrap.java:224)
    at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:289)
    at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159)
    at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150)
    at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86)
    at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124)
    at org.elasticsearch.cli.Command.main(Command.java:90)
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115)
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92)
Refer to the log for complete error details.

May I know how do I control the gc.log file is shared and writable to all ?

skyluke.1987 · June 14, 2019, 2:37am

[2019-06-14T10:22:26,473][INFO ][i.n.u.i.PlatformDependent] [eta10] Your platform does not provide complete low-level API for accessing direct buffers reliably. Unless explicitly requested, heap buffer will always be preferred to avoid potential system instability.
[2019-06-14T10:22:30,509][INFO ][o.e.x.s.a.s.FileRolesStore] [eta10] parsed [0] roles from file [/etc/elasticsearch-master/config/roles.yml]
[2019-06-14T10:22:30,794][INFO ][i.n.u.i.PlatformDependent] [eta10] Your platform does not provide complete low-level API for accessing direct buffers reliably. Unless explicitly requested, heap buffer will always be preferred to avoid potential system instability.
[2019-06-14T10:22:31,163][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [eta10] [controller/18298] [Main.cc@109] controller (64 bit): Version 7.1.0 (Build a8ee6de8087169) Copyright (c) 2019 Elasticsearch BV
[2019-06-14T10:22:31,519][DEBUG][o.e.a.ActionModule       ] [eta10] Using REST wrapper from plugin org.elasticsearch.xpack.security.Security
[2019-06-14T10:22:32,149][INFO ][o.e.d.DiscoveryModule    ] [eta10] using discovery type [zen] and seed hosts providers [settings]
[2019-06-14T10:22:33,116][INFO ][o.e.n.Node               ] [eta10] initialized
[2019-06-14T10:22:33,116][INFO ][o.e.n.Node               ] [eta10] starting ...
[2019-06-14T10:22:33,325][INFO ][o.e.t.TransportService   ] [eta10] publish_address {10.0.106.144:9300}, bound_addresses {10.0.106.144:9300}
[2019-06-14T10:22:33,339][INFO ][o.e.b.BootstrapChecks    ] [eta10] bound or publishing to a non-loopback address, enforcing bootstrap checks
ERROR: [1] bootstrap checks failed
[1]: initial heap size [2147483648] not equal to maximum heap size [32210157568]; this can cause resize pauses and prevents mlockall from locking the entire heap
[2019-06-14T10:22:33,351][INFO ][o.e.n.Node               ] [eta10] stopping ...
2019-06-14 10:22:33,356 pool-1-thread-1 ERROR Unable to unregister MBeans java.security.AccessControlException: access denied ("javax.management.MBeanServerPermission" "createMBeanServer")
        at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
        at java.security.AccessController.checkPermission(AccessController.java:884)
        at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
        at java.lang.management.ManagementFactory.getPlatformMBeanServer(ManagementFactory.java:465)
        at org.apache.logging.log4j.core.jmx.Server.unregisterLoggerContext(Server.java:248)
        at org.apache.logging.log4j.core.LoggerContext.stop(LoggerContext.java:340)
        at org.apache.logging.log4j.core.LoggerContext$1.run(LoggerContext.java:281)
        at org.apache.logging.log4j.core.util.DefaultShutdownCallbackRegistry$RegisteredCancellable.run(DefaultShutdownCallbackRegistry.java:109)
        at org.apache.logging.log4j.core.util.DefaultShutdownCallbackRegistry.run(DefaultShutdownCallbackRegistry.java:74)
        at java.lang.Thread.run(Thread.java:748)

Dear all, how should I resolve this? I have changed all the owner of ES to elasticsearch:elasticsearch. After I tried to start ./elasticsearch and I got this error.

May I know how do I verify? and correct it

skyluke.1987 · June 14, 2019, 2:51am

Hi all, I faced another issue as below:

pool-1-thread-1 ERROR Unable to unregister MBeans java.security.AccessControlException: access denied ("javax.management.MBeanServerPermission" "createMBeanServer")

Please provide me some support on this. Thanks

skyluke.1987 · June 14, 2019, 7:39am

May I know in this case I have to change the -Xms and -Xmx ?

Is there any guideline on how much should I put? Now I am putting at -Xms6g / -Xmx8g.
But it seems there is no changes.

ERROR: [1] bootstrap checks failed
[1]: initial heap size [2147483648] not equal to maximum heap size [32210157568]; this can cause resize pauses and prevents mlockall from locking the entire heap

Kindly assist .

dadoonet · June 15, 2019, 5:53am

Use the same value:

-Xms8g -Xmx8g

skyluke.1987 · June 18, 2019, 2:07am

Hi I have used this.

Maybe let me share the experience now. I have the old elasticsearch on my same server. Not sure if the file is still reading the old jvm file. May I know is there any config file is controlling this ?

dadoonet · June 18, 2019, 5:17am

But you wrote that you put

-Xms6g / -Xmx8g

vasek · June 18, 2019, 5:45am

You can try this to resolve the problem with permission:

chown -R elasticsearch: /usr/share/elasticsearch/ /etc/elasticsearch/ /var/lib/elasticsearch /var/log/elasticsearch

Directory structure depends on type of instalation (rpm, tarball,...) and you OS.

skyluke.1987 · June 18, 2019, 6:33am

Hi, but I have one problem. I have this root user, elasticsearch user and when I run I am using my own account which is user luke. How can I resolve and grant permissions to write and execute ?