How to setup a single node cluster for Elasticsearch 8.3 on Windows 10 on a laptop?

Elastic Stack Elasticsearch
1

Hello All,

I downloaded Elasticsearch 8.3 and Kibana 8.3 and I've run into lots of questions and issues about setting up a simple configuration to read a small CSV file. After running into some errors I started changing the defaults in the yaml file to:

# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: node-1
#
# Add custom attributes to the node:
#
node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
#path.data: /path/to/data
#
C:\elasticsearch-8.3.3\data
# Path to log files:
#
#path.logs: /path/to/logs
C:\elasticsearch-8.3.3\logs
#discovery.seed_hosts: ["host1", "host2"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
cluster.initial_master_nodes: ["node-1", "node-2"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# --------------------------------- Readiness ----------------------------------
#
# Enable an unauthenticated TCP readiness endpoint on localhost
#
readiness.port: 9399

So no matter how I try to set the path for data and logs, I keep getting this error and can't get past it:
Exception in thread "main" org.elasticsearch.common.settings.SettingsException: Failed to load settings from [elasticsearch.yml]
at org.elasticsearch.common.settings.Settings$Builder.loadFromStream(Settings.java:1194)
at org.elasticsearch.node.InternalSettingsPreparer.loadConfigWithSubstitutions(InternalSettingsPreparer.java:126)
at org.elasticsearch.node.InternalSettingsPreparer.prepareEnvironment(InternalSettingsPreparer.java:52)
at org.elasticsearch.common.cli.EnvironmentAwareCommand.createEnv(EnvironmentAwareCommand.java:110)
at org.elasticsearch.common.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:54)
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:85)
at org.elasticsearch.cli.Command.main(Command.java:50)
at org.elasticsearch.launcher.CliToolLauncher.main(CliToolLauncher.java:64)
Caused by: org.elasticsearch.xcontent.XContentParseException: [27:19] while scanning a simple key
in 'reader', line 35, column 1:
C:\elasticsearch-8.3.3\data
^
could not find expected ':'
in 'reader', line 36, column 1:
# Path to log files:

It was guesswork on my part, but it seemed to be a valid windows configuration and yes the paths are what I have on my machine. I've tried multiple ways to set the path to my data and logs, but nothing seems to work?

Does anyone know how to get this simple setup working?

2

Hi @JamesD_7 Welcome to the community

You need to have the

path.data in the setting... That is the field name... It's commented out in yours

#path.data

Should be

path.data: C:\elasticsearch-8.3.3\data

Same for the logs.

Technically it should run without changing any settings at all....

You don't need that readiness port either.

You have also put in multiple master nodes when you say you're trying to run one.

I would start over and not edit anything and then show us what your errors are having..

And then change a little as possible if you're new with elastic.

Did you follow the documents?

3

Yes, I was trying to follow the docs and no I'm not new to elastic, just new to this version (We used 5x and I've never set it up locally, the admin took care of the configuration so I focused just on queries.)

Here is what I got with the default yml file. Also, I put a small CSV file into the data folder and I assume I have to get Kibana started to load that manually? Not sure, but I'm not at that point yet and I never saw the terminal output for the username and password.

C:\elasticsearch-8.3.3>.\bin\elasticsearch.bat
[2022-08-22T05:50:13,560][INFO ][o.e.n.Node ] [DESKTOP-GOSPRBL] version[8.3.3], pid[10932], build[zip/801fed82df74dbe537f89b71b098ccaff88d2c56/2022-07-23T19:30:09.227964828Z], OS[Windows 10/10.0/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/18.0.2/18.0.2+9-61]
[2022-08-22T05:50:13,591][INFO ][o.e.n.Node ] [DESKTOP-GOSPRBL] JVM home [C:\elasticsearch-8.3.3\jdk], using bundled JDK [true]
[2022-08-22T05:50:13,591][INFO ][o.e.n.Node ] [DESKTOP-GOSPRBL] JVM arguments [-Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -Djava.security.manager=allow, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j2.formatMsgNoLookups=true, -Djava.locale.providers=SPI,COMPAT, --add-opens=java.base/java.io=ALL-UNNAMED, -XX:+UseG1GC, -Djava.io.tmpdir=C:\Users\jdrin\AppData\Local\Temp\elasticsearch, -XX:+HeapDumpOnOutOfMemoryError, -XX:+ExitOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Xms16353m, -Xmx16353m, -XX:MaxDirectMemorySize=8577351680, -XX:InitiatingHeapOccupancyPercent=30, -XX:G1ReservePercent=25, -Des.distribution.type=zip, --module-path=C:\elasticsearch-8.3.3\lib, -Djdk.module.main=org.elasticsearch.server]
[2022-08-22T05:50:25,039][INFO ][c.a.c.i.j.JacksonVersion ] [DESKTOP-GOSPRBL] Package versions: jackson-annotations=2.13.2, jackson-core=2.13.2, jackson-databind=2.13.2.2, jackson-dataformat-xml=2.13.2, jackson-datatype-jsr310=2.13.2, azure-core=1.27.0, Troubleshooting version conflicts: Troubleshoot dependency version conflicts when you use the Azure SDK for Java - Java on Azure | Microsoft Learn
[2022-08-22T05:50:30,140][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [aggs-matrix-stats]
[2022-08-22T05:50:30,140][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [analysis-common]
[2022-08-22T05:50:30,140][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [constant-keyword]
[2022-08-22T05:50:30,140][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [data-streams]
[2022-08-22T05:50:30,140][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [frozen-indices]
[2022-08-22T05:50:30,140][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [ingest-common]
[2022-08-22T05:50:30,140][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [ingest-geoip]
[2022-08-22T05:50:30,140][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [ingest-user-agent]
[2022-08-22T05:50:30,140][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [kibana]
[2022-08-22T05:50:30,140][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [lang-expression]
[2022-08-22T05:50:30,140][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [lang-mustache]
[2022-08-22T05:50:30,140][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [lang-painless]
[2022-08-22T05:50:30,140][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [legacy-geo]
[2022-08-22T05:50:30,140][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [mapper-extras]
[2022-08-22T05:50:30,140][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [mapper-version]
[2022-08-22T05:50:30,140][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [old-lucene-versions]
[2022-08-22T05:50:30,140][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [parent-join]
[2022-08-22T05:50:30,140][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [percolator]
[2022-08-22T05:50:30,140][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [rank-eval]
[2022-08-22T05:50:30,140][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [reindex]
[2022-08-22T05:50:30,140][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [repositories-metering-api]
[2022-08-22T05:50:30,140][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [repository-azure]
[2022-08-22T05:50:30,155][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [repository-encrypted]
[2022-08-22T05:50:30,155][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [repository-gcs]
[2022-08-22T05:50:30,155][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [repository-s3]
[2022-08-22T05:50:30,155][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [repository-url]
[2022-08-22T05:50:30,155][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [runtime-fields-common]
[2022-08-22T05:50:30,155][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [search-business-rules]
[2022-08-22T05:50:30,155][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [searchable-snapshots]
[2022-08-22T05:50:30,155][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [snapshot-based-recoveries]
[2022-08-22T05:50:30,155][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [snapshot-repo-test-kit]
[2022-08-22T05:50:30,155][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [spatial]
[2022-08-22T05:50:30,155][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [transform]
[2022-08-22T05:50:30,155][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [transport-netty4]
[2022-08-22T05:50:30,155][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [unsigned-long]
[2022-08-22T05:50:30,171][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [vector-tile]
[2022-08-22T05:50:30,171][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [vectors]
[2022-08-22T05:50:30,171][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [wildcard]
[2022-08-22T05:50:30,171][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [x-pack-aggregate-metric]
[2022-08-22T05:50:30,171][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [x-pack-analytics]
[2022-08-22T05:50:30,171][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [x-pack-async]
[2022-08-22T05:50:30,171][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [x-pack-async-search]
[2022-08-22T05:50:30,171][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [x-pack-autoscaling]
[2022-08-22T05:50:30,176][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [x-pack-ccr]
[2022-08-22T05:50:30,176][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [x-pack-core]
[2022-08-22T05:50:30,176][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [x-pack-deprecation]
[2022-08-22T05:50:30,176][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [x-pack-enrich]
[2022-08-22T05:50:30,176][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [x-pack-eql]
[2022-08-22T05:50:30,176][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [x-pack-fleet]
[2022-08-22T05:50:30,176][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [x-pack-graph]
[2022-08-22T05:50:30,176][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [x-pack-identity-provider]
[2022-08-22T05:50:30,176][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [x-pack-ilm]
[2022-08-22T05:50:30,176][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [x-pack-logstash]
[2022-08-22T05:50:30,176][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [x-pack-ml]
[2022-08-22T05:50:30,176][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [x-pack-monitoring]
[2022-08-22T05:50:30,176][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [x-pack-ql]
[2022-08-22T05:50:30,176][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [x-pack-rollup]
[2022-08-22T05:50:30,176][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [x-pack-security]
[2022-08-22T05:50:30,176][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [x-pack-shutdown]
[2022-08-22T05:50:30,176][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [x-pack-sql]
[2022-08-22T05:50:30,176][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [x-pack-stack]
[2022-08-22T05:50:30,176][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [x-pack-text-structure]
[2022-08-22T05:50:30,176][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [x-pack-voting-only-node]
[2022-08-22T05:50:30,176][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] loaded module [x-pack-watcher]
[2022-08-22T05:50:30,176][INFO ][o.e.p.PluginsService ] [DESKTOP-GOSPRBL] no plugins loaded
[2022-08-22T05:50:32,639][INFO ][o.e.e.NodeEnvironment ] [DESKTOP-GOSPRBL] using [1] data paths, mounts [[(C:)]], net usable_space [1tb], net total_space [1.8tb], types [NTFS]
[2022-08-22T05:50:32,639][INFO ][o.e.e.NodeEnvironment ] [DESKTOP-GOSPRBL] heap size [15.9gb], compressed ordinary object pointers [true]
[2022-08-22T05:50:32,824][INFO ][o.e.n.Node ] [DESKTOP-GOSPRBL] node name [DESKTOP-GOSPRBL], node ID [lFoFr-5cSoCvGTvH48jSHg], cluster name [elasticsearch], roles [data_hot, ml, data_frozen, ingest, data_cold, data, remote_cluster_client, master, data_warm, data_content, transform]
[2022-08-22T05:50:35,085][INFO ][o.e.i.g.DatabaseNodeService] [DESKTOP-GOSPRBL] deleting stale file [C:\Users\jdrin\AppData\Local\Temp\elasticsearch\geoip-databases\lFoFr-5cSoCvGTvH48jSHg\GeoLite2-ASN.mmdb]
[2022-08-22T05:50:35,085][INFO ][o.e.i.g.DatabaseNodeService] [DESKTOP-GOSPRBL] deleting stale file [C:\Users\jdrin\AppData\Local\Temp\elasticsearch\geoip-databases\lFoFr-5cSoCvGTvH48jSHg\GeoLite2-ASN.mmdb_COPYRIGHT.txt]
[2022-08-22T05:50:35,085][INFO ][o.e.i.g.DatabaseNodeService] [DESKTOP-GOSPRBL] deleting stale file [C:\Users\jdrin\AppData\Local\Temp\elasticsearch\geoip-databases\lFoFr-5cSoCvGTvH48jSHg\GeoLite2-ASN.mmdb_elastic-geoip-database-service-agreement-LICENSE.txt]
[2022-08-22T05:50:35,085][INFO ][o.e.i.g.DatabaseNodeService] [DESKTOP-GOSPRBL] deleting stale file [C:\Users\jdrin\AppData\Local\Temp\elasticsearch\geoip-databases\lFoFr-5cSoCvGTvH48jSHg\GeoLite2-ASN.mmdb_LICENSE.txt]
[2022-08-22T05:50:35,085][INFO ][o.e.i.g.DatabaseNodeService] [DESKTOP-GOSPRBL] deleting stale file [C:\Users\jdrin\AppData\Local\Temp\elasticsearch\geoip-databases\lFoFr-5cSoCvGTvH48jSHg\GeoLite2-City.mmdb]
[2022-08-22T05:50:35,101][INFO ][o.e.i.g.DatabaseNodeService] [DESKTOP-GOSPRBL] deleting stale file [C:\Users\jdrin\AppData\Local\Temp\elasticsearch\geoip-databases\lFoFr-5cSoCvGTvH48jSHg\GeoLite2-City.mmdb_COPYRIGHT.txt]
[2022-08-22T05:50:35,101][INFO ][o.e.i.g.DatabaseNodeService] [DESKTOP-GOSPRBL] deleting stale file [C:\Users\jdrin\AppData\Local\Temp\elasticsearch\geoip-databases\lFoFr-5cSoCvGTvH48jSHg\GeoLite2-City.mmdb_elastic-geoip-database-service-agreement-LICENSE.txt]
[2022-08-22T05:50:35,101][INFO ][o.e.i.g.DatabaseNodeService] [DESKTOP-GOSPRBL] deleting stale file [C:\Users\jdrin\AppData\Local\Temp\elasticsearch\geoip-databases\lFoFr-5cSoCvGTvH48jSHg\GeoLite2-City.mmdb_LICENSE.txt]
[2022-08-22T05:50:35,101][INFO ][o.e.i.g.DatabaseNodeService] [DESKTOP-GOSPRBL] deleting stale file [C:\Users\jdrin\AppData\Local\Temp\elasticsearch\geoip-databases\lFoFr-5cSoCvGTvH48jSHg\GeoLite2-City.mmdb_README.txt]
[2022-08-22T05:50:35,101][INFO ][o.e.i.g.DatabaseNodeService] [DESKTOP-GOSPRBL] deleting stale file [C:\Users\jdrin\AppData\Local\Temp\elasticsearch\geoip-databases\lFoFr-5cSoCvGTvH48jSHg\GeoLite2-Country.mmdb]
[2022-08-22T05:50:35,101][INFO ][o.e.i.g.DatabaseNodeService] [DESKTOP-GOSPRBL] deleting stale file [C:\Users\jdrin\AppData\Local\Temp\elasticsearch\geoip-databases\lFoFr-5cSoCvGTvH48jSHg\GeoLite2-Country.mmdb_COPYRIGHT.txt]
[2022-08-22T05:50:35,101][INFO ][o.e.i.g.DatabaseNodeService] [DESKTOP-GOSPRBL] deleting stale file [C:\Users\jdrin\AppData\Local\Temp\elasticsearch\geoip-databases\lFoFr-5cSoCvGTvH48jSHg\GeoLite2-Country.mmdb_elastic-geoip-database-service-agreement-LICENSE.txt]
[2022-08-22T05:50:35,101][INFO ][o.e.i.g.DatabaseNodeService] [DESKTOP-GOSPRBL] deleting stale file [C:\Users\jdrin\AppData\Local\Temp\elasticsearch\geoip-databases\lFoFr-5cSoCvGTvH48jSHg\GeoLite2-Country.mmdb_LICENSE.txt]
[2022-08-22T05:50:35,348][INFO ][o.e.x.s.Security ] [DESKTOP-GOSPRBL] Security is enabled
[2022-08-22T05:50:35,744][INFO ][o.e.x.s.a.s.FileRolesStore] [DESKTOP-GOSPRBL] parsed [0] roles from file [C:\elasticsearch-8.3.3\config\roles.yml]
[2022-08-22T05:50:36,245][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [DESKTOP-GOSPRBL] [controller/17844] [Main.cc@123] controller (64 bit): Version 8.3.3 (Build d2d2e518384d45) Copyright (c) 2022 Elasticsearch BV
[2022-08-22T05:50:36,816][INFO ][o.e.t.n.NettyAllocator ] [DESKTOP-GOSPRBL] creating NettyAllocator with the following configs: [name=elasticsearch_configured, chunk_size=1mb, suggested_max_allocation_size=1mb, factors={es.unsafe.use_netty_default_chunk_and_page_size=false, g1gc_enabled=true, g1gc_region_size=8mb}]
[2022-08-22T05:50:36,847][INFO ][o.e.i.r.RecoverySettings ] [DESKTOP-GOSPRBL] using rate limit [40mb] with [default=40mb, read=0b, write=0b, max=0b]
[2022-08-22T05:50:36,885][INFO ][o.e.d.DiscoveryModule ] [DESKTOP-GOSPRBL] using discovery type [multi-node] and seed hosts providers [settings]
[2022-08-22T05:50:38,088][INFO ][o.e.n.Node ] [DESKTOP-GOSPRBL] initialized
[2022-08-22T05:50:38,088][INFO ][o.e.n.Node ] [DESKTOP-GOSPRBL] starting ...
[2022-08-22T05:50:38,250][INFO ][o.e.x.s.c.f.PersistentCache] [DESKTOP-GOSPRBL] persistent cache index loaded
[2022-08-22T05:50:38,250][INFO ][o.e.x.d.l.DeprecationIndexingComponent] [DESKTOP-GOSPRBL] deprecation component started
[2022-08-22T05:50:38,435][INFO ][o.e.t.TransportService ] [DESKTOP-GOSPRBL] publish_address {127.0.0.1:9300}, bound_addresses {127.0.0.1:9300}, {[::1]:9300}
[2022-08-22T05:50:39,242][WARN ][o.e.b.BootstrapChecks ] [DESKTOP-GOSPRBL] the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured
[2022-08-22T05:50:39,242][WARN ][o.e.b.BootstrapChecks ] [DESKTOP-GOSPRBL] Transport SSL must be enabled if security is enabled. Please set [xpack.security.transport.ssl.enabled] to [true] or disable security by setting [xpack.security.enabled] to [false]
[2022-08-22T05:50:39,242][INFO ][o.e.c.c.ClusterBootstrapService] [DESKTOP-GOSPRBL] this node is locked into cluster UUID [EBPdujMLS56yCv70lczT8w] and will not attempt further cluster bootstrapping
[2022-08-22T05:50:39,273][INFO ][o.e.c.c.ClusterBootstrapService] [DESKTOP-GOSPRBL] no discovery configuration found, will perform best-effort cluster bootstrapping after [3s] unless existing master is discovered
[2022-08-22T05:50:39,643][INFO ][o.e.c.s.MasterService ] [DESKTOP-GOSPRBL] elected-as-master ([1] nodes joined)[FINISH_ELECTION, {DESKTOP-GOSPRBL}{lFoFr-5cSoCvGTvH48jSHg}{3yO7kb8USbqlulCGEMxQHQ}{DESKTOP-GOSPRBL}{127.0.0.1}{127.0.0.1:9300}{cdfhilmrstw} completing election], term: 3, version: 35, delta: master node changed {previous , current [{DESKTOP-GOSPRBL}{lFoFr-5cSoCvGTvH48jSHg}{3yO7kb8USbqlulCGEMxQHQ}{DESKTOP-GOSPRBL}{127.0.0.1}{127.0.0.1:9300}{cdfhilmrstw}]}
[2022-08-22T05:50:39,859][INFO ][o.e.c.s.ClusterApplierService] [DESKTOP-GOSPRBL] master node changed {previous , current [{DESKTOP-GOSPRBL}{lFoFr-5cSoCvGTvH48jSHg}{3yO7kb8USbqlulCGEMxQHQ}{DESKTOP-GOSPRBL}{127.0.0.1}{127.0.0.1:9300}{cdfhilmrstw}]}, term: 3, version: 35, reason: Publication{term=3, version=35}
[2022-08-22T05:50:39,944][INFO ][o.e.h.AbstractHttpServerTransport] [DESKTOP-GOSPRBL] publish_address {127.0.0.1:9200}, bound_addresses {127.0.0.1:9200}, {[::1]:9200}
[2022-08-22T05:50:39,944][INFO ][o.e.n.Node ] [DESKTOP-GOSPRBL] started {DESKTOP-GOSPRBL}{lFoFr-5cSoCvGTvH48jSHg}{3yO7kb8USbqlulCGEMxQHQ}{DESKTOP-GOSPRBL}{127.0.0.1}{127.0.0.1:9300}{cdfhilmrstw}{xpack.installed=true, ml.machine_memory=34295828480, ml.max_jvm_size=17154703360}
[2022-08-22T05:50:40,229][INFO ][o.e.l.LicenseService ] [DESKTOP-GOSPRBL] license [6eb59865-ca66-4e0e-8c0f-b8f8c5317fcc] mode [basic] - valid
[2022-08-22T05:50:40,229][INFO ][o.e.x.s.a.Realms ] [DESKTOP-GOSPRBL] license mode is [basic], currently licensed security realms are [reserved/reserved,file/default_file,native/default_native]
[2022-08-22T05:50:40,229][INFO ][o.e.g.GatewayService ] [DESKTOP-GOSPRBL] recovered [1] indices into cluster_state
[2022-08-22T05:50:40,576][ERROR][o.e.i.g.GeoIpDownloader ] [DESKTOP-GOSPRBL] exception during geoip databases update
org.elasticsearch.ElasticsearchException: not all primary shards of [.geoip_databases] index are active
at org.elasticsearch.ingest.geoip.GeoIpDownloader.updateDatabases(GeoIpDownloader.java:134) ~[ingest-geoip-8.3.3.jar:8.3.3]
at org.elasticsearch.ingest.geoip.GeoIpDownloader.runDownloader(GeoIpDownloader.java:274) [ingest-geoip-8.3.3.jar:8.3.3]
at org.elasticsearch.ingest.geoip.GeoIpDownloaderTaskExecutor.nodeOperation(GeoIpDownloaderTaskExecutor.java:102) [ingest-geoip-8.3.3.jar:8.3.3]
at org.elasticsearch.ingest.geoip.GeoIpDownloaderTaskExecutor.nodeOperation(GeoIpDownloaderTaskExecutor.java:48) [ingest-geoip-8.3.3.jar:8.3.3]
at org.elasticsearch.persistent.NodePersistentTasksExecutor$1.doRun(NodePersistentTasksExecutor.java:42) [elasticsearch-8.3.3.jar:?]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:769) [elasticsearch-8.3.3.jar:?]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) [elasticsearch-8.3.3.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
at java.lang.Thread.run(Thread.java:833) [?:?]
[2022-08-22T05:50:42,613][INFO ][o.e.c.r.a.AllocationService] [DESKTOP-GOSPRBL] current.health="GREEN" message="Cluster health status changed from [RED] to [GREEN] (reason: [shards started [[.geoip_databases][0]]])." previous.health="RED" reason="shards started [[.geoip_databases][0]]"
[2022-08-22T05:50:43,120][INFO ][o.e.i.g.DatabaseNodeService] [DESKTOP-GOSPRBL] successfully loaded geoip database file [GeoLite2-Country.mmdb]
[2022-08-22T05:50:43,183][INFO ][o.e.i.g.DatabaseNodeService] [DESKTOP-GOSPRBL] successfully loaded geoip database file [GeoLite2-ASN.mmdb]
[2022-08-22T05:50:43,919][INFO ][o.e.i.g.DatabaseNodeService] [DESKTOP-GOSPRBL] successfully loaded geoip database file [GeoLite2-City.mmdb]

//When I check to see if Elasticsearch is running, I get a dialog prompt for username and password, however, that info never loaded in
the terminal window like the docs mentioned.
http://localhost:9200

4

Hi Couple things

  1. Those are all normal logs including some of those exception but in the end

Cluster health status changed from [RED] to [GREEN]

  1. That is because you already set it up at one time... and it came up once before... at this point you can 1 of 2 things

A) Which is what I would suggest: Completely delete all the directories / software etc especially the elasticsearch data directories which should be under the same directory and start from scratch.. using this guide

B) Reset the elastic password see here

FInally if you use the defaults elasticsearch will run on https so that would be

https://localhost:9200

And typically you would us curl to access that see here

curl --cacert %ES_HOME%\config\certs\http_ca.crt -u elastic https://localhost:9200

5

Well I made it to the command prompt information. The I tried to verify that Elasticsearch was running with that curl command.

-> Elasticsearch security features have been automatically configured!
-> Authentication is enabled and cluster connections are encrypted.
-> Password for the elastic user (reset with bin/elasticsearch-reset-password -u elastic):
..........DsFYy
-> HTTP CA certificate SHA-256 fingerprint: etc...

I deleted the previous characters for the password just to be extra safe even though it appears to be randomly generated.

// I ran this from a command prompt window as admin:
C:\WINDOWS\system32>curl --cacert %ES_HOME%\config\certs\http_ca.crt -u elastic https://localhost:9200
Enter host password for user 'elastic': //Put in the full password here that I got.
curl: (77) schannel: failed to open CA file '%ES_HOME%\config\certs\http_ca.crt': The system cannot find the path specified.

// I verified that Elastic setup the certs folder with the certs file:
C:\elasticsearch-8.3.3\config\certs
I tried to run Kibana next from a command prompt (another window, not as admin)with:
.\bin\kibana.bat
However, it just seems to hang there. My kibana is standalone that I unzipped to C:\kibana-8.3.3 and ran the command from that path.

So am I doing the wrong steps here or I am I missing something else again?

6

I don't know what that means...

%ES_HOME% mean you set the environment variable to the base path where you unzipped elasticsearch, common windows syntax so if you did not set that Env variable then you need to the full path the cert which will be under where you unzipped it all.

Lets get Elastic working first.. then worry about kibana...

7

I don't know what that means...I just meant that I didn't post the entire password that was generated.

I didn't set any windows environment variables for Elastic. I tried this with curl in windows 10 as it's builtin with my windows version:

C:\WINDOWS\system32>curl --cacert C:\elasticsearch-8.3.3\config\certs\http_ca.crt -u elastic https://localhost:9200
Enter host password for user 'elastic':
curl: (60) schannel: CertGetCertificateChain trust error CERT_TRUST_REVOCATION_STATUS_UNKNOWN
More details here: curl - SSL CA Certificates

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

Then I thought I would just verify curl really is installed on windows, which it is.

C:\WINDOWS\system32>curl.exe -V
curl 7.83.1 (Windows) libcurl/7.83.1 Schannel
Release-Date: 2022-05-13
Protocols: dict file ftp ftps http https imap imaps pop3 pop3s smtp smtps telnet tftp
Features: AsynchDNS HSTS IPv6 Kerberos Largefile NTLM SPNEGO SSL SSPI UnixSockets

So, I don't think I entered the wrong password from the error, but I'm not sure what the error is telling me.

The good news, I guess, is I know exactly what files/folders to delete each time I want to rerun Elasticsearch from the last failed attempt.

So after I get Elasticsearch started without errors, which is where I'm at now, what is the next step? Is it verify that it's running with curl like I've been trying?

Lets get Elastic working first.. Agreed. I'm taking small steps here.

8

Yes

you can try one of these it will not validate the cert just to see if it is running

curl --insecure -u elastic https://localhost:9200

curl -k -u elastic https://localhost:9200

Then if it is running run this... notice small -v

curl -v --cacert C:\elasticsearch-8.3.3\config\certs\http_ca.crt -u elastic https://localhost:9200

9

Hello Stephen,
I verified that Elasticsearch is running, but it fails on the cert? Here is output from the terminal window:

Microsoft Windows [Version 10.0.19044.1889]
(c) Microsoft Corporation. All rights reserved.

C:\WINDOWS\system32>curl --insecure -u elastic https://localhost:9200
Enter host password for user 'elastic':
{
"name" : "DESKTOP-GOSPRBL",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "lK7dbiaCThqi_FH9ieotcQ",
"version" : {
"number" : "8.3.3",
"build_flavor" : "default",
"build_type" : "zip",
"build_hash" : "801fed82df74dbe537f89b71b098ccaff88d2c56",
"build_date" : "2022-07-23T19:30:09.227964828Z",
"build_snapshot" : false,
"lucene_version" : "9.2.0",
"minimum_wire_compatibility_version" : "7.17.0",
"minimum_index_compatibility_version" : "7.0.0"
},
"tagline" : "You Know, for Search"
}

C:\WINDOWS\system32>curl -v --cacert C:\elasticsearch-8.3.3\config\certs\http_ca.crt -u elastic https://localhost:9200
Enter host password for user 'elastic':

  • Trying 127.0.0.1:9200...
  • Connected to localhost (127.0.0.1) port 9200 (#0)
  • schannel: disabled automatic use of client certificate
  • ALPN: offers http/1.1
  • schannel: added 1 certificate(s) from CA file 'C:\elasticsearch-8.3.3\config\certs\http_ca.crt'
  • schannel: CertGetCertificateChain trust error CERT_TRUST_REVOCATION_STATUS_UNKNOWN
  • Closing connection 0
    curl: (60) schannel: CertGetCertificateChain trust error CERT_TRUST_REVOCATION_STATUS_UNKNOWN
    More details here: curl - SSL CA Certificates

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

I went to that link and found this:

If libcurl was built with Schannel (Microsoft's native TLS engine) or Secure Transport (Apple's native TLS engine) support, then libcurl will still perform peer certificate verification, but instead of using a CA cert bundle, it will use the certificates that are built into the OS.

So I disabled the cert check and got in:

C:\WINDOWS\system32>curl -v --ssl-no-revoke --cacert C:\elasticsearch-8.3.3\config\certs\http_ca.crt -u elastic https://localhost:9200
Enter host password for user 'elastic':

  • Trying 127.0.0.1:9200...
  • Connected to localhost (127.0.0.1) port 9200 (#0)
  • schannel: disabled automatic use of client certificate
  • ALPN: offers http/1.1
  • schannel: added 1 certificate(s) from CA file 'C:\elasticsearch-8.3.3\config\certs\http_ca.crt'
  • schannel: connection hostname (localhost) validated against certificate name (localhost)
  • ALPN: server did not agree on a protocol. Uses default.
  • Server auth using Basic with user 'elastic'

GET / HTTP/1.1
Host: localhost:9200
Authorization: Basic ZWxhc3RpYzpkQXJqX0hBdD1xdVNlNUZIRmtCVw==
User-Agent: curl/7.83.1
Accept: /

  • Mark bundle as not supporting multiuse
    < HTTP/1.1 200 OK
    < X-elastic-product: Elasticsearch
    < content-type: application/json
    < content-length: 538
    <
    {
    "name" : "DESKTOP-GOSPRBL",
    "cluster_name" : "elasticsearch",
    "cluster_uuid" : "lK7dbiaCThqi_FH9ieotcQ",
    "version" : {
    "number" : "8.3.3",
    "build_flavor" : "default",
    "build_type" : "zip",
    "build_hash" : "801fed82df74dbe537f89b71b098ccaff88d2c56",
    "build_date" : "2022-07-23T19:30:09.227964828Z",
    "build_snapshot" : false,
    "lucene_version" : "9.2.0",
    "minimum_wire_compatibility_version" : "7.17.0",
    "minimum_index_compatibility_version" : "7.0.0"
    },
    "tagline" : "You Know, for Search"
    }
  • Connection #0 to host localhost left intact

Then I started up Kibana and I got this:

C:>cd kibana-8.3.3

C:\kibana-8.3.3>.\bin\kibana.bat
[2022-08-23T12:08:12.016-05:00][INFO ][plugins-service] Plugin "cloudSecurityPosture" is disabled.
[2022-08-23T12:08:12.093-05:00][INFO ][http.server.Preboot] http server running at http://localhost:5601
[2022-08-23T12:08:12.313-05:00][INFO ][plugins-system.preboot] Setting up [1] plugins: [interactiveSetup]
[2022-08-23T12:08:12.317-05:00][INFO ][preboot] "interactiveSetup" plugin is holding setup: Validating Elasticsearch connection configuration…
[2022-08-23T12:08:12.355-05:00][INFO ][root] Holding setup until preboot stage is completed.

i Kibana has not been configured.

Go to http://localhost:5601/?code=796176 to get started.

I went to that url in Firefox and pasted the enrollment token that Elasticsearch generated, which opened up a dialog login. I used: elastic as the username and I used the generated password that worked for the command prompts. However, I got "Username or password is incorrect. Please try again." So what appears to be wrong with this?

10

Cool Good Progress

Don't know...You got a typo?....

Did you install kibana from scratch again? If not I would...

and you can generate an enrollment token see here

.\bin\elasticsearch-create-enrollment-token -s kibana --url "https://localhost:9200"

11

No, but I realized I needed to so as I was getting some errors. I'm in the process of doing so. I don't think I did a typo on the password, but I'll try it again.

Also, at the kibana login dialog, it's expecting elastic as the username, correct?

12

Correct

You are close...

13

Hello Stephen,
Yes you were correct about putting a clean Kibana version back out. I got into the integrations page and started to install logstash, which I thought would fit my use case? It is, ingest a small csv file in order to run queries against it.

So my questions are: for this specific use case, is logstash the best integration module? If so, where do I put the policy file? Per the docs:

Copy this policy to the elastic-agent.yml on the host where the Elastic Agent is installed.

but I can't find an elastic-agent.yml file in Elastic or Kibana?

The intent is to keep it simple, yet functional, and it appears I've failed the simple part. This is my only use case for a while, the only difference is some of the files will have a lot more rows, but still just in the thousands.

Aftermath:
Okay, after much more reading I found this link: Analyzing Aftershock Data with Elasticsearch and Kibana via CSV Import | Elastic Blog and the Data Visualizer feature, which is drag n drop. Worked great, so now I'm configuring things to run queries. I should be good to go now.

Thank you,
James

1 Like
14

@JamesD_7 Awesome Glad you are up and running!

And yes you can do a lot with data visualizer

Open a new Thread when you have more questions!

Lots of ways to ingest Data

  • Via REST API
  • Data Visualizer
  • Beats
  • Logstash (BTW logstash does not use a policy file so a little mixing of stuff on that)
  • Elastic Agent
15

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.