Easiest OS to install ELK 7.x on?

Elastic Stack Elasticsearch
1

Hello,

Really NEW user here, and also really new or have forgotten most of my LINUX Skills. Anyway, I tried setting up a VM running CENTOS version 7 and then installed JAVA as instructions stated. Then I installed Elasticsearch, but that's as far as I have gotten. It took me forever to realize that all of the folder permissions wer set such that I could not even list directory contents; in some cases, even root was not granted permission. After I got permissions right, I verified that the JAVA HOME and CLASSPATH were properly exported and visible / set, but I continue to get errors and the service never starts. It tells me if I disable "Machine Learning" the service should start, but I have set that parameter to false in the config file, and still no dice. I have spent 16+ hours on this one and was just wondering if ANY particular OS or LINUX version works best or is easiest to configure in your opinions...

Thanks !

2

I use Elasticsearch on CentOS 7 and am honestly very surprised to hear that you have spent 16+ hours on making Elasticsearch work and it still doesn't. In my experience it just works. No permissions issues, no messing with config (unless you're setting up a cluster spread over multiple servers of course :wink: ). I'm also surprised by your mention of JAVA HOME or CLASSPATH, as I've never needed to check or mess with them. I just make sure Java is installed by way of installing java-1.8.0-openjdk-headless from the CentOS 7 repos.

How did you install Elasticsearch?

If you install Elasticsearch from the rpm it sets up a service and then all you should need to run Elasticsearch is start the service.

If you unpack the tgz then all you should need to do is invoke bin/elasticsearch in the directory it unpacked to as whatever user you unpacked the tgz as.

It might help if you posted your errors.

3

I installed via yum which appeared to grab the rpm. After I tried to start the service and it would not start, I started googling and one site said specifically - after installing Java, to export the Env Variables for Java such that it would be able to find Java. I think it was an unnecessary step, since it obviously found Java to begin with. After install, the ".yml" ? Config file had all entries commented out like host, port, and such, so I went into it and removed the #'s and replaced them with the defaults. Still no joy. Could be the Java Heap size, maybe ? All I got initially was that it was unable to run the Machine Learning Modules, to set the parameter to "false" and try restarting... I saw lots of folks online with the permission issues, i.e. could not view the logfiles, could not even list the contents of the elasticsearch log folder. Had to do chmod's on most of the folders to see any of the log or config data...

Will try to grab and post errors this evening. Got to do the day job at the moment :-0

4

you don't use yum to grab rpm.
Try downloading latest rpm from elk's website and use rpm -Uvh to install them.

Then next step is to go through all yml file. you can't just uncomment blindly everything.
read up little on comment section and it will make sense.

if you do java -version on command line and if you find java that means elasticsearch is going to find as well.

5

Why did you do that? Did you try starting the elasticsearch service before you started editing things? I suspect editing the config files is where your problems started. I advise you try uninstalling the rpm, make sure /etc/elasticsearch/ /var/log/elasticsearch/ and /usr/share/elasticsearch/ are all gone. Then make sure/usr/bin/java exists and points at a working Java install, which it should if you installed it Java from CentOS repos, then re-install the elasticsearch rpm and try starting the service.
$ systemctl start elasticsearch.service
Wait a minute then check
$ systemctl status elasticsearch.service
If it says everything is OK try
$ curl http://localhost:9200
If you don't get a response and systemctl says the service is still running, wait a minute then try again. Depending on your machine specs Elasticsearch may not have started up to the point you can communicate with it. If all you want is a single instance of Elasticsearch that doesn't talk to any other instances, it really should just work without you messing with anything at all.

Some people seem to see any permissions which prevent them doing something how they want or expect to be able to do it as broken. Like they find they can't view the contents of /var/log/elasticsearch/ as a non-root user and even though it's very common for such logs to only be viewable by root, they see that as broken. So they change permissions and/or ownership to "fix" that. And some of the people who decide they have to change permissions and/or ownership of the logs do so in a way which prevents what needs to write to the logs from doing so. So in fixing something that wasn't broken they have actually broken it and maybe they realise they've broken it or maybe they don't and then they start doing other things which make things even worse.

6

Thank-you for your comments. I did not just blindly un-comment everything, but dang, one would assume that IP and Port needed to be un-commented, but maybe the default uses the IP of the machine and its name and the default port. I used the directions here: https://darrenoneill.eu/?p=660 - this guy used YUM and appeared to have no issues. As noted, blind LINUX novice here. Anyway, after disabling the Machine Learning as the initial error recommended, I get the Error Log below. Java cannot allocate enough memory for some reason. Machine has 4 Gigs total at the moment. Also unable to bind to Port and IP? Log:

[2020-02-10T15:44:21,516][WARN ][o.e.b.JNANatives ] [CENTOS7] Unable to lock JVM Memory: error=12, reason=Cannot allocate memory
[2020-02-10T15:44:21,537][WARN ][o.e.b.JNANatives ] [CENTOS7] This can result in part of the JVM being swapped out.
[2020-02-10T15:44:21,538][WARN ][o.e.b.JNANatives ] [CENTOS7] Increase RLIMIT_MEMLOCK, soft limit: 65536, hard limit: 65536
[2020-02-10T15:44:21,539][WARN ][o.e.b.JNANatives ] [CENTOS7] These can be adjusted by modifying /etc/security/limits.conf, for example:
# allow user 'elasticsearch' mlockall
elasticsearch soft memlock unlimited
elasticsearch hard memlock unlimited
[2020-02-10T15:44:21,539][WARN ][o.e.b.JNANatives ] [CENTOS7] If you are logged in interactively, you will have to re-login for the new limits to take effect.
[2020-02-10T15:44:22,359][INFO ][o.e.e.NodeEnvironment ] [CENTOS7] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [33.2gb], net total_space [35.1gb], types [rootfs]
[2020-02-10T15:44:22,362][INFO ][o.e.e.NodeEnvironment ] [CENTOS7] heap size [1007.3mb], compressed ordinary object pointers [true]
[2020-02-10T15:44:22,366][INFO ][o.e.n.Node ] [CENTOS7] node name [CENTOS7], node ID [bG_yMTJaSoqmO5GY5WSE8w], cluster name [elasticsearch]
[2020-02-10T15:44:22,367][INFO ][o.e.n.Node ] [CENTOS7] version[7.5.2], pid[1149], build[default/rpm/8bec50e1e0ad29dad5653712cf3bb580cd1afcdf/2020-01-15T12:11:52.313576Z], OS[Linux/3.10.0-1062.el7.x86_64/amd64], JVM[AdoptOpenJDK/OpenJDK 64-Bit Server VM/13.0.1/13.0.1+9]
[2020-02-10T15:44:22,368][INFO ][o.e.n.Node ] [CENTOS7] JVM home [/usr/share/elasticsearch/jdk]
[2020-02-10T15:44:22,369][INFO ][o.e.n.Node ] [CENTOS7] JVM arguments [-Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=COMPAT, -Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Djava.io.tmpdir=/tmp/elasticsearch-4885883396744010983, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=/var/lib/elasticsearch, -XX:ErrorFile=/var/log/elasticsearch/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=/var/log/elasticsearch/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -XX:MaxDirectMemorySize=536870912, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/etc/elasticsearch, -Des.distribution.flavor=default, -Des.distribution.type=rpm, -Des.bundled_jdk=true]
[2020-02-10T15:44:28,470][INFO ][o.e.p.PluginsService ] [CENTOS7] loaded module [aggs-matrix-stats]
[2020-02-10T15:44:28,472][INFO ][o.e.p.PluginsService ] [CENTOS7] loaded module [analysis-common]
[2020-02-10T15:44:28,472][INFO ][o.e.p.PluginsService ] [CENTOS7] loaded module [flattened]
[2020-02-10T15:44:28,473][INFO ][o.e.p.PluginsService ] [CENTOS7] loaded module [frozen-indices]
......

[2020-02-10T15:44:42,258][INFO ][o.e.n.Node ] [CENTOS7] initialized
[2020-02-10T15:44:42,258][INFO ][o.e.n.Node ] [CENTOS7] starting ...
[2020-02-10T15:44:42,688][ERROR][o.e.b.Bootstrap ] [CENTOS7] Exception
org.elasticsearch.transport.BindTransportException: Failed to bind to [9300-9400]
at org.elasticsearch.transport.TcpTransport.bindToPort(TcpTransport.java:389) ~[elasticsearch-7.5.2.jar:7.5.2]
at org.elasticsearch.transport.TcpTransport.bindServer(TcpTransport.java:355) ~[elasticsearch-7.5.2.jar:7.5.2]
at org.elasticsearch.transport.netty4.Netty4Transport.doStart(Netty4Transport.java:135) ~[?:?]
at org.elasticsearch.xpack.core.security.transport.netty4.SecurityNetty4Transport.doStart(SecurityNetty4Transport.java:81) ~[?:?]
at org.elasticsearch.xpack.security.transport.netty4.SecurityNetty4ServerTransport.doStart(SecurityNetty4ServerTransport.java:43) ~[?:?]
at org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:59) ~[elasticsearch-7.5.2.jar:7.5.2]
at org.elasticsearch.transport.TransportService.doStart(TransportService.java:230) ~[elasticsearch-7.5.2.jar:7.5.2]
at org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:59) ~[elasticsearch-7.5.2.jar:7.5.2]
at org.elasticsearch.node.Node.start(Node.java:688) ~[elasticsearch-7.5.2.jar:7.5.2]
at org.elasticsearch.bootstrap.Bootstrap.start(Bootstrap.java:293) ~[elasticsearch-7.5.2.jar:7.5.2]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:378) [elasticsearch-7.5.2.jar:7.5.2]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) [elasticsearch-7.5.2.jar:7.5.2]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) [elasticsearch-7.5.2.jar:7.5.2]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) [elasticsearch-7.5.2.jar:7.5.2]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:125) [elasticsearch-cli-7.5.2.jar:7.5.2]
at org.elasticsearch.cli.Command.main(Command.java:90) [elasticsearch-cli-7.5.2.jar:7.5.2]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115) [elasticsearch-7.5.2.jar:7.5.2]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) [elasticsearch-7.5.2.jar:7.5.2]
Caused by: java.net.BindException: Cannot assign requested address
at sun.nio.ch.Net.bind0(Native Method) ~[?:?]
at sun.nio.ch.Net.bind(Net.java:469) ~[?:?]
at sun.nio.ch.Net.bind(Net.java:458) ~[?:?]
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:220) ~[?:?]
at io.netty.channel.socket.nio.NioServerSocketChannel.doBind(NioServerSocketChannel.java:134) ~[?:?]
at io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:551) ~[?:?]
at io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1346) ~[?:?]
at io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:503) ~[?:?]
at io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:488) ~[?:?]
at io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:985) ~[?:?]
at io.netty.channel.AbstractChannel.bind(AbstractChannel.java:247) ~[?:?]
at io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:344) ~[?:?]
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) ~[?:?]
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:510) ~[?:?]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:518) ~[?:?]
at io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1050) ~[?:?]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
at java.lang.Thread.run(Thread.java:830) ~[?:?]
[2020-02-10T15:44:42,735][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [CENTOS7] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: BindTransportException[Failed to bind to [9300-9400]]; nested: BindException[Cannot assign requested address];
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-7.5.2.jar:7.5.2]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-7.5.2.jar:7.5.2]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.5.2.jar:7.5.2]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:125) ~[elasticsearch-cli-7.5.2.jar:7.5.2]
at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-7.5.2.jar:7.5.2]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115) ~[elasticsearch-7.5.2.jar:7.5.2]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-7.5.2.jar:7.5.2]
Caused by: org.elasticsearch.transport.BindTransportException: Failed to bind to [9300-9400]
at org.elasticsearch.transport.TcpTransport.bindToPort(TcpTransport.java:389) ~[elasticsearch-7.5.2.jar:7.5.2]
at org.elasticsearch.transport.TcpTransport.bindServer(TcpTransport.java:355) ~[elasticsearch-7.5.2.jar:7.5.2]
at org.elasticsearch.transport.netty4.Netty4Transport.doStart(Netty4Transport.java:135) ~[?:?]
at org.elasticsearch.xpack.core.security.transport.netty4.SecurityNetty4Transport.doStart(SecurityNetty4Transport.java:81) ~[?:?]
at org.elasticsearch.xpack.security.transport.netty4.SecurityNetty4ServerTransport.doStart(SecurityNetty4ServerTransport.java:43) ~[?:?]
at org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:59) ~[elasticsearch-7.5.2.jar:7.5.2]
at org.elasticsearch.transport.TransportService.doStart(TransportService.java:230) ~[elasticsearch-7.5.2.jar:7.5.2]
at org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:59) ~[elasticsearch-7.5.2.jar:7.5.2]
at org.elasticsearch.node.Node.start(Node.java:688) ~[elasticsearch-7.5.2.jar:7.5.2]
at org.elasticsearch.bootstrap.Bootstrap.start(Bootstrap.java:293) ~[elasticsearch-7.5.2.jar:7.5.2]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:378) ~[elasticsearch-7.5.2.jar:7.5.2]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.5.2.jar:7.5.2]
... 6 more
Caused by: java.net.BindException: Cannot assign requested address
at sun.nio.ch.Net.bind0(Native Method) ~[?:?]
at sun.nio.ch.Net.bind(Net.java:469) ~[?:?]
at sun.nio.ch.Net.bind(Net.java:458) ~[?:?]
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:220) ~[?:?]
at io.netty.channel.socket.nio.NioServerSocketChannel.doBind(NioServerSocketChannel.java:134) ~[?:?]
at io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:551) ~[?:?]
at io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1346) ~[?:?]
at io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:503) ~[?:?]
at io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:488) ~[?:?]
at io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:985) ~[?:?]
at io.netty.channel.AbstractChannel.bind(AbstractChannel.java:247) ~[?:?]
at io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:344) ~[?:?]
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) ~[?:?]
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:510) ~[?:?]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:518) ~[?:?]
at io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1050) ~[?:?]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
at java.lang.Thread.run(Thread.java:830) [?:?]
[2020-02-10T15:44:43,315][INFO ][o.e.n.Node ] [CENTOS7] stopping ...
[2020-02-10T15:44:43,323][INFO ][o.e.n.Node ] [CENTOS7] stopped
[2020-02-10T15:44:43,323][INFO ][o.e.n.Node ] [CENTOS7] closing ...
[2020-02-10T15:44:43,369][INFO ][o.e.n.Node ] [CENTOS7] closed

7

Ok, you guys have been great and I will document your suggestions as I try to learn two pieces of software at the same time or maybe I should say 4 (ELK + CENTOS LINUX). Made recommended Memory allocation changes, corrected an IP Address "fat finger" and then I had to set the discovery.type to single-node. Service says it's started now.... Ughh on to Logstash next.

Thanks Much !

8

logstash will be complete different learning curve now. enjoy

9

If one assumed that, one would be wrong. :wink: The blog post you cite says nothing about editing the config file, which is maybe the only good thing I could think to say about it. It was written on 30th June 2018. It shows installing Elasticsearch 1.7.3 which went End Of Life on 16th January 2017. It shows service management commands that you'd use on a system with init.d (CentOS 6) rather than systemd (CentOS 7). (Those commands will work, but better to use systemctl on CentOS 7.) At the end it says

 sudo wegt localhost:9200
 cat index.html

wegt is obviously a typo but ignoring that, everything about that is bizarre.

Glad to hear you got it working at last. With Logstash, try changing nothing initially unless Elastic's documentation says to. :wink:

10

Thanks Mike ! Love it when someone teaches me something... It will always come back around to ya Sir !

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.