ELK stack in windows 11

Elastic Stack Logstash
1

I tried to install ELK stack, Elasticsearch, Logstash, Kibana. Elasticsearch and Kibana run succesfully, but logstash returned some exceptions. How can I fix it?
This is last logs from logstash :

[2023-04-26T16:45:34,397][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2023-04-26T16:45:37,532][INFO ][logstash.config.source.local.configpathloader] No config files found in path {:path=>"D:/Programming/mkb/ELK/logstash-8.7.0/logstash.conf"}
[2023-04-26T16:45:37,534][ERROR][logstash.config.sourceloader] No configuration found in the configured sources.
[2023-04-26T16:45:37,813][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false}
[2023-04-26T16:45:37,827][INFO ][logstash.runner          ] Logstash shut down.
[2023-04-26T16:45:37,833][FATAL][org.logstash.Logstash    ] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:790) ~[jruby.jar:?]
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:753) ~[jruby.jar:?]
        at D_3a_.Programming.mkb.ELK.logstash_minus_8_dot_7_dot_0.lib.bootstrap.environment.<main>(D:\Programming\mkb\ELK\logstash-8.7.0\lib\bootstrap\environment.rb:91) ~[?:?]
2

The error says that no configuration was found.

What is the content of the file: D:/Programming/mkb/ELK/logstash-8.7.0/logstash.conf?

3

Hi, I'm new in ELK stack, so I can't find how to change D:/Programming/mkb/ELK/logstash-8.7.0/logstash.conf this path ?
My logstash.conf file:

input {
    file{
      path => "D:/Programming/mkb/ELK/Software/elk-example/logs/elk-stack.log"
      start_position => "beginning"
  }
}

output {
  stdout {
    codec => rubydebug  
  }
  
  elasticsearch {
    hosts => ["https://localhost:9200"]
	index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
  }
}
4

My logstash.conf file is situated in D:/Programming/mkb/ELK/logstash-8.7.0/bin/logstash.conf

5

And I also run with this code in terminal
logstash -f D:/Programming/mkb/ELK/logstash-8.7.0/bin/logstash.conf
But it also didn't work, I don't know why

6 
Sending Logstash logs to D:/Programming/mkb/ELK/logstash-8.7.0/logs which is now configured via log4j2.properties
[2023-04-26T17:01:09,977][INFO ][logstash.runner          ] Log4j configuration path used is: D:\Programming\mkb\ELK\logstash-8.7.0\config\log4j2.properties
[2023-04-26T17:01:09,995][WARN ][logstash.runner          ] The use of JAVA_HOME has been deprecated. Logstash 8.0 and later ignores JAVA_HOME and uses the bundled JDK. Running Logstash with the bundled JDK is recommended. The bundled JDK has been verified to work with each specific version of Logstash, and generally provides best performance and reliability. If you have compelling reasons for using your own JDK (organizational-specific compliance requirements, for example), you can configure LS_JAVA_HOME to use that version instead.
[2023-04-26T17:01:09,999][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"8.7.0", "jruby.version"=>"jruby 9.3.10.0 (2.6.8) 2023-02-01 107b2e6697 OpenJDK 64-Bit Server VM 17.0.6+10 on 17.0.6+10 +indy +jit [x86_64-mswin32]"}
[2023-04-26T17:01:10,003][INFO ][logstash.runner          ] JVM bootstrap flags: [-Xms1g, -Xmx1g, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true, -Djruby.regexp.interruptible=true, -Djdk.io.File.enableADS=true, --add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED, --add-opens=java.base/java.security=ALL-UNNAMED, --add-opens=java.base/java.io=ALL-UNNAMED, --add-opens=java.base/java.nio.channels=ALL-UNNAMED, --add-opens=java.base/sun.nio.ch=ALL-UNNAMED, --add-opens=java.management/sun.management=ALL-UNNAMED]
[2023-04-26T17:01:10,102][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2023-04-26T17:01:12,600][ERROR][logstash.config.sourceloader] No configuration found in the configured sources.
[2023-04-26T17:01:12,768][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false}
[2023-04-26T17:01:12,791][INFO ][logstash.runner          ] Logstash shut down.
[2023-04-26T17:01:12,799][FATAL][org.logstash.Logstash    ] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:790) ~[jruby.jar:?]
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:753) ~[jruby.jar:?]
        at D_3a_.Programming.mkb.ELK.logstash_minus_8_dot_7_dot_0.lib.bootstrap.environment.<main>(D:\Programming\mkb\ELK\logstash-8.7.0\lib\bootstrap\environment.rb:91) ~[?:?]

These are all of my logs from logstash

7

It is the same error, it can't find any configuration.

How are you starting logstash? Please share the command you are using.

8

logstash -f D:/Programming/mkb/ELK/logstash-8.7.0/bin/logstash.conf

9

Where is really your file? In your previous error you tried to use this configuration in this path:

"D:/Programming/mkb/ELK/logstash-8.7.0/logstash.conf"

But now you are running it as it was in another directory, D:/Programming/mkb/ELK/logstash-8.7.0/bin/logstash.conf.

You need to check where the file really is an pass the correct path.

10

it is real path,D:/Programming/mkb/ELK/logstash-8.7.0/bin/logstash.conf
but when I run logstash, it returned me warning log that

No config files found in path {:path=>"D:/Programming/mkb/ELK/logstash-8.7.0/logstash.conf"}

11

Well, something is not right, this means that you are running logstash this way:

logstash -f D:/Programming/mkb/ELK/logstash-8.7.0/logstash.conf

But if your file is in another directory, you need to run this way:

logstash -f D:/Programming/mkb/ELK/logstash-8.7.0/bin/logstash.conf

I would also use double quotes between the path, so try this:

logstash -f "D:/Programming/mkb/ELK/logstash-8.7.0/bin/logstash.conf"
12

Thank you, I found another way, I moved config file to base url ),
But I came across next problem, The elasticsearch's logs returned me some warning that

http client did not trust this server's certificate, closing connection;
But I configured it yml file

# Enable security features
xpack.security.enabled: true

xpack.security.enrollment.enabled: true

# Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents
xpack.security.http.ssl:
  enabled: true
  keystore.path: D:/Programming/mkb/ELK/elasticsearch-8.7.0/config/certs/http.p12

# Enable encryption and mutual authentication between cluster nodes
xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: D:/Programming/mkb/ELK/elasticsearch-8.7.0/config/certs/transport.p12
  truststore.path: D:/Programming/mkb/ELK/elasticsearch-8.7.0/config/certs/transport.p12

How can I fix it ?

13

You need to share the entire log, not just part of it, is impossible to know where this is from.

14 
[2023-04-26T17:42:15,082][INFO ][o.e.n.Node               ] [node-1] started {node-1}{5zTRuBvYQkCYFJ0hu2FG5A}{m7SQRj7BSyWcDb8xrfZ3yg}{node-1}{127.0.0.1}{127.0.0.1:9300}{cdfhilmrstw}{8.7.0}{ml.machine_memory=21246033920, xpack.installed=true, ml.allocated_processors_double=8.0, ml.max_jvm_size=10628366336, ml.allocated_processors=8}
[2023-04-26T17:42:15,540][INFO ][o.e.l.LicenseService     ] [node-1] license [6da674d3-4bc6-4c35-82c4-70be1ab39ea0] mode [basic] - valid
[2023-04-26T17:42:15,543][INFO ][o.e.x.s.a.Realms         ] [node-1] license mode is [basic], currently licensed security realms are [reserved/reserved,file/default_file,native/default_native]
[2023-04-26T17:42:15,543][INFO ][o.e.g.GatewayService     ] [node-1] recovered [10] indices into cluster_state
[2023-04-26T17:42:17,354][INFO ][o.e.h.n.s.HealthNodeTaskExecutor] [node-1] Node [{node-1}{5zTRuBvYQkCYFJ0hu2FG5A}] is selected as the current health node.
[2023-04-26T17:42:19,606][INFO ][o.e.c.r.a.AllocationService] [node-1] current.health="GREEN" message="Cluster health status changed from [RED] to [GREEN] (reason: [shards started [[.apm-source-map][0]]])." previous.health="RED" reason="shards started [[.apm-source-map][0]]"
[2023-04-26T17:44:49,526][WARN ][o.e.h.n.Netty4HttpServerTransport] [node-1] http client did not trust this server's certificate, closing connection Netty4HttpChannel{localAddress=/[0:0:0:0:0:0:0:1]:9200, remoteAddress=/[0:0:0:0:0:0:0:1]:7820}
15

You need to configure it in logstash, you need to add a cacert setting pointing to the CA that you used to generate elasticsearch certs.

Also, if your input is not beat, you should not use that index name as the field @metadata will not exist.

  elasticsearch {
      hosts => ["https://localhost:9200"]
      index => "logstash-%{+YYYY.MM.dd}"
      cacert => "/path/to/the/ca.crt"
  }

Try something like this if it does not work I recommend that you open a different topic, but first read this documentation and the logstash documentation.

1 Like
16

Thank you so much bro

17

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.